M27 Milestone Updates

Cost optimization. In the last milestone, we deployed a custom pod scheduler and used it for our add-ons. We have now extended its use to general-purpose dynos as well. This resulted in significant cost savings and optimal use of computing resources in the cluster. Memory usage ranges between 70-80% (previously 30-40), and CPU usage is around 50%. CPU usage is maintained low by design so that applications that have not opted for a dedicated CPU can share this leftover CPU.

Our Docker images are now hosted in the AWS Elastic Container Registry (ECR). We have been using a Registry hosted in-house in our cluster so far. We had provisioned a 4TB EBS volume for container image storage and had used 60% of it. Also, the computing resources required to run the registry server were increasing. So, we compared our in-house cost to ECR, a fully managed solution and found it cost-effective. In addition to the cost savings, we now have an Image Registry that is scalable and more stable. Occasional image pull errors that we used to get have stopped.

Build caching: Once we enabled layer caching in our build system, application build times (after the first build) dropped by 35-45%.

More control over the application's computing resources and cost: We introduced a slider control for users to choose the amount of CPU and RAM their dynos need. We observed that CPU usage is generally low across the board except on a few dynos. Now, we have introduced a "Dedicated CPU" checkbox for dynos and add-ons. If unchecked, the dyno will share the CPU from a shared pool. If your dyno needs dedicated CPU usage, you can switch anytime.

Sleep-when-idle was a cost-saving feature available on all review apps. Now, you can enable this feature on any app by checking a box in the application settings.

We use Cloudnative Buildpacks for our build system. We have been using open-source buildpacks as they are. We need more control over our build system to make the onboarding experience for new apps smoother, optimize the build process for time and resources, and reduce the size of the generated machine image. We are now learning the internals of Buildpacks so that we can build our own system or customize the existing one. In addition to the benefits mentioned above, we must streamline our support for complex custom dependencies and build steps that some applications demand. This project is at a POC level and will span the coming milestones.

We have added a new buildpack called neeto-deploy/prebuild to our build system. This buildpack executes the neeto-deploy-prestart.sh file present in the source code before starting the build process. This helps to set dynamic environment variables or execute a custom logic before the build.