Embracing The Cloud
Cloud architecture is a great way to ship safe, cost effective, resilient, and scalable applications.
Here are some tips from the trenches to help your team embrace the cloud.
Trust Your Cloud Provider
If you are going to be using the cloud, you need to trust that your provider’s services are going to be operational. Writing code defensively or testing that a particular service is working is a waste of time. Let your services fail. As long as your monitoring strategy is strong, you’ll know when an issue arises.
Embrace Asynchronicity
AWS Lambda Functions are triggered by events – HTTP, SQS, Cron, SNS, Streams – to name a few. Being prepared to reason about your services/applications from an asynchronous perspective will help in the long run. Let this thinking permeate throughout the full software development life cycle.
Use Real Infrastructure
Don’t make the mistake of “creating” a personal cloud. There are tools out there that help you do so, and it may seem attractive, but there is no substitute for a real cloud. If you’ve fully embraced the cloud, which I hope you do, you’re likely using things that cannot be faked reliably. Let real infrastructure influence the way you develop locally from the beginning.
Write Tests
Do write tests, and write them against real infrastructure when integrating.
Figure 1: The Testing Pyramid
- Many Unit Test
- Some Service Tests
- Few UI Tests (acceptance/regression)
No one knows every edge of a language or framework. Testing is a good practice to increase quality, validate your assumptions, and avoid regression.
Logging & Monitoring
A Lambda Function may trigger other services, which may trigger more Lambda Functions. This graph of events is hard to keep track of if your logging implementation is off. Write your functions in such a way that you get some helpful logging by default (think middleware). Pay attention to what you are logging and at what level (error, warning, debug, info) to reduce noise in a production environment.
Use a third party tool for monitoring to reduce the amount of things you have to maintain in the cloud. Epsagon has been a great choice so far.
Measuring Performance
Think about how you will measure the performance of your services. Ask these questions:
- Is running your acceptance tests in parallel by some factor an acceptable coverage area?
- What is the baseline for response times?
- Should notifications be configured for long responses?
Know The Common Limits And Pitfalls Of Your Tools
Designing infrastructure? Script it with CloudFormation or Terraform.
Know what happens when you change the properties of a cloud resource. Does it drop the resource and recreate it? How do you handle that destruction? Are you keeping backups of your data (DB, File Storage, etc.) on a regular basis?
Design a Deployment Policy for Production
Developers should not be allowed to deploy (ship infrastructure) into production from their local environments. Make this so by using IAM or similar. Each developer should have their own account and unable to use service accounts that run in CI/CD.
Use Elastic Search For Searching DynamoDB
Using Global Secondary Indices is nice for known searches, but having a robust way to search a DynamoDB table is preferred. Embrace the cloud’s event based nature by using DynamoDB Streams to keep Elastic Search indices up-to-date. Write easy to use utilities, that trigger events which use the same stream functions for reindexing.
Ship Frequently
Your service or application should be able to be shipped at any time (CI/CD). The more frequent you ship, the fewer chances for things to go wrong. Don’t let HUGE features bog down your deployment cadence. Break them down to small, shippable parts. If your logging, monitoring, and testing are in order, you will know if something is broken before a user experiences it.
What are you shipping?
Are you shipping to the cloud? If you need help, drop me a line or visit my services page for more information.