I received an email this morning way after midnight that made me think of a quote by Otto Von Bismarck “Laws are like sausages. To retain respect for sausages and laws, one must not watch them in the making.” The implication is that ignorance is bliss. That may be true for sausages and laws, but it is definitely not true for e-commerce.
At Accorin, we are very transparent and collaborative in strategy, design, development and quality assurance processes, but usually don’t share too much about the development operations (DevOps) that goes on behind the scenes to ensure that engineers are productive and we have a controlled Development to Staging to Production deployment process for our clients. As part of our DevOps process we regularly update our development environments (Docker stacks) when Magento comes out with a new release. As with making sausage and laws, it can be a bit more difficult than expected.
Here is an email from a DevOps Manager explaining the behind the scenes process; something that most of our clients don’t see but definitely benefit from:
”I just finished the setup and demo for the latest M2 Demo Server. While setting it up I also setup the Docker Environments for all developers to work on Client. This setup will be used for all 2.4 updates.
This took longer than expected. In the end about 8 hours total (most are around 2 hours). The good news is that for any new projects on 2.4, I have all of the code/scripts/etc. all committed in this project for reuse. We can simply use the scripts to kick them off very quickly.
Here are the issues that took the extra time:
- rabbitmq is now required for all Commerce deployments, this was fairly quick. Using 3.8 since 3.7 is EOL end of 9/2020.
- elasticsearch is now required, and 6.5 is EOL, so we are now using 6.8 on anything new and will need to move to 6.8 on other projects in the near future. This is a good update but was more difficult. It was fairly easy to setup the local stack with this, but there isn’t a configuration for elasticsearch available in env.php. So this has to be done with database value overrides in env.php. This was quick for the local stack but took some time for deployments, all done now.
- The Elasticsearch connection must be present (just like redis with 2.3) during build and setup for deployments. So this means that I have to setup temporary ES instances during build like I do with redis during CI deployment, this is much easier with redis since redis is so lightweight, but ES uses over 1.5 GB of memory just to launch. It also required several linux sysctl changes in linux on the docker stack and gitlab which is a known issue with ES and its large requirements just to run. (https://github.com/elastic/elasticsearch-docker/issues/92) Once these tweaks were done, that worked fine.
- Magento 2.4 is no longer compatible with Amazon RDS Aurora 5.6 (1.22.2) and so I had to upgrade our Development RDS server to 5.7 (2.0.8). This was trivial to do but to do this major upgrade I had to actually delete the old database and make a snapshot and then build a new one using the snapshot and took some time.
- One of the most time consuming issues was a new plugin Magento is using for composer setup of new projects called the Magento/composer-root-update-plugin. It can be found here: https://github.com/Magento/composer-root-update-plugin/
This plug caused some issues but I read all of the docs and found that this plugin is really only needed upon upgrading, so it can be removed. https://devdocs.Magento.com/guides/v2.3/comp-mgr/cli/upgrade-with-plugin.html
All of that said, all is working now and seems to be working well. I didn’t expect it to take this long but I don’t recall that last Magento release that had this many changes to system requirements. Let me know if you have any questions.”
The Bottom Line
As with so much in e-commerce, a lot of work happens behind the scenes and, like making sausages, sometimes you just want the sausage and don’t need to know what goes into it, but sometimes it is good to know the ingredients and work that the Accorin team puts in to make our customers’ sites run smoothly.
And finally, if you are a sausage fan, rest easy. Sausage making today is a lot more precise than making laws. Here is a great article on it.
