This is so easy that anyone can do this. Why do we need an architect?
To start building a new cloud solution is a very easy job. Just google or bing for a tutorial on how to create it in your preferred cloud. This is why I sometimes have discussions with clients and project teams on whether an architect is even needed in a project team. Almost every time the answer would be that there is a need for the architect role.
To understand the benefits of having an architect in team, let’s talk a bit about what architecture is for and what a good architect brings to the project.
Purpose of architecture
Architecture is the overall design and implementation guidelines for the information system. This purpose is the same in the new cloud era as it was in the old on-premise world. Architecture focuses on the impactful and "hard to change later" -aspects of the system.
The "hard to change" -aspect has shifted hugely in the cloud era. The old hard to change aspect was hardware and software investments. With cloud services and zero investment this is the easy part now. The scalability and running costs are the new hard part, in addition to the manageability. Manageability has been important and will be even more important with the wide portfolio of cloud services used. Without good cloud architecture the complexity will ruin your system development velocity and you are not able to respond to new business requirements agilely. Without proper flexible architecture you could even get hurt by cloud service provider deprecating some old cloud service that you still depend on. I have heard this happened...
Purpose of cloud architecture is to make solution development and management easier and to provide more business value with better scalability and flexibility now and in the future.
Architect does the planning of the system so that it fulfills the owner's requirements and fits for its purpose now and in the future. Sometimes this plan can be done at a very high and abstract level, sometimes it needs to be in a very detailed level with concrete development guidelines.
If the project already has detailed plans that are verified to match client requirements and just need to be executed, then there is no role for an architect. In data and analytics related software projects this is very rarely the situation. Normal projects in this area are either implementing something totally new, or a mix of something existing and something new. In these scenarios a cloud architect should be in the team to guide decisions on what cloud services should be used and how. Otherwise the implementation will be done without technical leadership. you will end up somewhere, not neccessary there where you wanted to be.
Developer vs. architect
You need the best team of superstar developers to implement a system that a decent architect could have designed so that any team with normal developers could have implemented it easily.
There is a clear difference between developer and architect roles. Developer focuses on implementing the plan right. Architect focuses on implementing the right plan. Developer has skills to do a specific implementation. Architect has knowledge and wisdom for finding out, choosing and planning the way to implement. Developer focuses on how, architect on why. Both roles are necessary.
Some developers have already much experience and can also do architectural planning, and some architects are able to work hands-on and do the development. It’s not either-or, it is a continuum. Architect with hands-on experience is valuable also for guiding the detailed development work.
Pitfalls for cloud architecture
There are some pitfalls to avoid in modern cloud architecture.
Rely on best practices. Best practice documentations are nice, but sometimes you should create new even better practices. Even if someone states a best practice, it could not be that in your case. Do not blindly rely on best practices without understanding them.
Reusing the good old architecture. Things change, especially in the cloud. Do not rely in the old practices without validating. Especially do not reuse your on-premise best practices and architectures without validating them and modernizing to fit cloud.
Not knowing the requirements. If you do not know what the requirements are now and in the future, you cannot plan for fulfilling them. Sometimes the initial requirements can be very conflicting, e.g. "we want realtime, scalable and cheap system with 100% SLA". There is a big impact on technical architecture depending on the definition of realtime. People can talk about realtime system and mean that the data should be less than 1 hour fresh, or the data should be less than 10ms fresh. Make sure you know the requirements and know where you are aiming to. If you do not know where you are going, you may end up exactly there.
Aiming directly for the final architecture. Architecture is and should be evolving in cloud. Architecture used to be about big, hard to change decisions. In the cloud architecture is about designing things so that they are possible to change when needed. The flexibility can be an important requirement for example when planning for scalable architecture. Implement a target architecture plan and roadmap with steps to get there. Keep your target architecture up-to-date.
Impact of good architecture
Good architecture makes it easy to implement systems. It also makes it easy to maintain and develop further these systems. These are very important in agile cloud era.
Architecture is not any more about "hard to change" -topics like in on-premise world, because in cloud there is no "hard to change" -topics like huge hardware or license investments! Good Cloud architecture is more about flexibility to change things when necessary. This can have huge impact on enabling new innovations quickly. You do not want to be fixed to your current setup for next five years in cloud and analytics world. Things progress so quickly that it is a requirement to be able to utilise new cutting edge technologies all the time.
Good Architecture is like salt. If you have too much of it, it tastes bad. If you have too little, it does not taste at all. With a good amount of architecture, the system works deliciously and nobody even notices the perfect architecture.