Dear AWS, how do I build & develop purely on AWS right now?

The announcements from AWS around deprecating certain services have raised a bunch of questions and concerns in the AWS community. 

As Jeff Barr wrote, these are the services:

S3 Select, CloudSearch, Cloud9, SimpleDB, Forecast, Data Pipeline, and CodeCommit.

This post will focus on Cloud9 and CodeCommit … and how I think this announcement impacts the “end to end” developer story for developers on AWS. We’ll also look at how the announcements impacts my “Go-To” services Amazon CodeCatalyst.

It is written from the perspective of a builder that mainly uses AWS tools for smaller side projects and can be seen as a “startup” that needs to quickly be up & running without much hassle.

Introduction

These annoucements, and the way that these deprecations where announced

Blog for CodeCommit

Blog for Cloud9

are in my humble opinion one of the worst possible ways. I know the teams at AWS have seen the feedback and I hope that there will be a clearer communication strategy going forward.

For me the combination of those posts with the assumption of CodeCatalyst being built on top of these services gives a very strange feeling on how much AWS is currently invested into Developers on AWS.

Let’s look at why I see a lot of impact of these announcements for builders and think about alternatives if you are using CodeCommit or Cloud9 for certain aspects today.

Tools required for SDLC

A few weeks ago I even dedicated a complete Shorts-Playlist on all of the Code* tools, looking at their usage and the approach to cover a full Software Development Lifecycle (SDLC) building on AWS.

In this series I drafted the diagram:

AWS Tools part of your SDLC until recent announcements

This being and end-to-end flow, AWS had at least two options to implement this process using their tools:

Either CodeCatalyst or a combination of different AWS Services

When CodeCatalyst was announced, I wrote about how CodeCatalyst can be used to cover all parts of your Secure Development Lifecycle (SDLC) process on AWS. Ever since then, there was an alternative on AWS using a combination of different building blocks: CodeCommit, CodeBuild, CodeDeploy, CodePipeline and others.

CodeCommit was a good, reliable managed Git server. For the purposes it solved, there weren’t many features to add. It was a managed service you didn’t need to think about and just “serve its purpose”.

Cloud9 was a hosted IDE, a development environment that users where able to access through their browser. This enabled builders to have a real IDE, even on old or underpowered computers, anywhere — even when being on vacation.

Developers on AWS are still able to use CodeCatalyst to cover for all parts of your product lifecycle or they had the alternative to use the different “building blocks” to compose their SDLC process. Both options gave value and helped AWS customers to solve certain aspects and problems.

Now, officially, only one option is left — CodeCatalyst.
CodeCatalyst is an integrated DevTools service that unites all of the building blocks under an opinionated, structured user interface. It was announced at re:Invent 2022 and went GA in early 2023. With the custom blueprints feature, it also enables builders to create project templates and share them with their team mates or dependend teams. Very powerful possibilities for teams to collaborate better and also share their best practices with other teams.

Those that didn’t need a “reliable managed Git server” where most probably using existing alternatives — that might solve the “job” better than CodeCommit — like Github, Gitlab or Atlassian. These users and AWS customers are not affected by the change.

What has changed with the July 2024 announcements — builders perspective

Now, the system landscape has changed.

Developers can not use Cloud9 anymore to develop software, they need to fallback to alternatives like Github Codespaces, Coder or Gitpod.

Developers cannot store their source code in CodeCommit anymore, they need to fallback to alternatives like Github, Gitlab or Bitbucket.

And given CodeCatalyst might be using CodeCommit under the hood and is using Cloud9 for the DevEnvironments – Can I really build something on top of CodeCatalyst going forward?

So this announcement of deprecation — without a “real” AWS native alternative — puts everyone building and developing software on AWS in the situation of needed to look for alternative setups.

Especially it forces you — if you are a small organization (or a startup) to engage with more than just one vendor as part of your SDLC lifecycle and process. I see this as a critical point to talk about aswell.

And, if you are building software or platforms on AWS where especially CodeCommit is part of application or the deployed architecture itself — You are now left without any option. If you want to integrate a Git server in your application on AWS, you will now need to self-host the git-server instead of using a managed service.

If you “just” needed a Git repository, quickly, fast and reliable — CodeCommit was the way to go. Now, you need to use a 3rd party alternative.

Now: What options on AWS do we have as builders?

What changed with the July 2024 announcements — business perspective

Looking at the announced changes from a different perspective, we need to acknowledge that AWS is a 90+ billion (90.000.000.000) dollar company. It is clear, being a business that aims to “make money”, that AWS needs to focus on services and solutions that are widely used, adapted and earn a good margin.

The reason might be that Cloud9 and CodeCommit where just not profitable enough to drive the expected growth of the business. Especially, as there are other services that do the same job better than Cloud9 and CodeCommit. So it might have been “just” a business decision to stop investing into these services and focus instead on Amazon Q that promises to help developers and builders on AWS.

This raises the question on which other services might soon or in the future be hit by exactly the same challenge. And – how is “success” measured by AWS on services? Is it “just” revenue or are there other points that are being considered?

But still — How this feels for me and questions I have (emotionally)

It feels like AWS has given up the game of engaging with their “Builders” and is now focused on the “Buyers” that “host” their applications on AWS.

If you think about how AWS started and if you look at how much effort AWS has spent this year on making us think that “Amazon Q Developer” is going to make our lives as developers easier…

How can I as an advocate for AWS as a platform be confident that I am valued as “Builder” on AWS? Will other services also disappear if they do not get enough traction?

And how much can I trust in Werner’s “Now, go build“?

How much “trust” can I put in the other Code* (CodeBuild, CodePipeline, …) tools on AWS?
With CodePipeline and CodeBuild getting a lot of notable updates right now (macOS, Github action runners, Stages rollback, …) the outsiders view is that at least these services are there to exist… but how much trust has the AWS team lost with Builders around the globe?

I’m eager to see how the different workshops, best practice documents, open source projects that use either CodeCommit oder Cloud9 (especially also AWS owned) will be adjusted and updated in the next weeks and months.

How much is also CodeCatalyst going to be the central place for Developers on AWS? How much updates will we see there?

How does this affect you – I would love to know!

I am really interested to hear how these announcements have affected your perspective on AWS and your view on the different AWS services.

Please share your thoughts either as a comment to this post or reach out to me personally!

What YOU can do next

You could now follow the advice from AWS and “migrate” away from CodeCommit or Cloud9 — but is this really what you want to do?
If you have a need to have a “Git server” or “Git repository” close to your applications on AWS, how do you do that?
You might need to host your own Git server on AWS….or you need to give up on that premise and fallback to alternative Git providers like Github, Gitlab, …

If you insist on having your own hosted Git within your AWS environment, there a few possible solutions…

…and potentially others that I am not aware of….

In order to host a “simple” Git setup I’ve recently made this repository public that deploys Gitness as a Git Repository on ECS. It will cost you roughly 50 USD/month. See also a relevant blog post.
Inspired by this, Jakub Wolynko did the same thing for Onedev – please see https://github.com/3sky/onedev-on-ecs if you would like to try that out.

As an alternative for Cloud9, you can use vscode.dev, which runs VS Code in the browser or other alternatives that are more integrated and personalized like gitpod.io or Github Codespaces.

But is this REALLY what you want to do if you are working on AWS only?

What I hope to get from the AWS team

As re:Invent is approaching fast and that usually sets the direction for a lot of AWS services, I really hope to get reliable information and roadmap clarifications around the AWS developer tools.

I’d like to understand if I can rely on CodeCatalyst, CodePipeline, CodeBuild, CodeArtifact, CodeDeploy, … and other AWS services that help developers to build software on AWS.

Does anyone know if this page ever mentioned CodeCatalyst? Please let me know!

In addition to that, I would love to get a better and more detailed overview on what the level of support will be that customers of the “deprecated” services will get: Security Updates? Priority Support?
Creating one page that summerizes that for all “deprecated” services would be amazing!

And – last but not least – make sure that Amazong Q knows which services you are deprecating!

Screenshot taken on 6th of September, 4pm CEST

If you’ve read this post until here, I would love to get your view and your feedback on this topic!

Thanks for the feedback I got before publishing this article and while I know you don’t agree with everything I wrote, it’s great to get your feedback, Monika, Raphael, Ran, Markus and others 🙂

Please let me know either in the comments or directly on my social channels — LinkedIn, X being the ones I still use mostly 😉 

Views: 221

A self-hosted CodeCommit alternative

A few weeks ago AWS CodeCommit became a deprecated service on AWS. This means, customers cannot create new repositories anymore – refer to this announcement for all details: Blog for CodeCommit

There are obviously a lot of alternatives to CodeCommit (Github, Gitlab, …) but if you need a “self-hosted” Git repository in your own AWS account this can become a little bit harder to provision.

If you’re looking for a “self-hosted” Git server, a bunch of tools come up:

  • Gitea
  • Gitness
  • Gitlab
  • …and obviously also others or just a “git” server running on EC2

As I wanted to be able to deploy something that works “out of the box” I looked at how to provision one of these alternatives on my own AWS account.

Gitness

Gitness is an open source development platform packed with the power of code hosting and automated DevOps pipelines. It’s biggest mantainer is Harness. It includes “way more” than just Git – eg Gitspaces, Pipelines, etc.

Deployment of Gitness

If you want to deploy Gitness, the docs point you at running it locally using Docker, or by deploying it on EC2 or k8s.

My aim was to “only” make the “Git” component available and because of that I’ve chosen Amazon ECS with EFS as storage to provision Gitness.

Open Source Code for the deployment of Gitness

I’ve set up a project on Github where all of the code examples are available for you to look at:

https://github.com/Lock128/setup-gitness

In the rest of the article, I’ll try to walk you through the most important parts of the project.

Please note that this code is not production-ready, it is more a PoC to showcase the direction that you could take.

Source Overview

We’re using AWS CDK and Typescript for Infrastructure As Code (IaC).

We have a “bin” directory where the main application is and a “lib” directory where the “Cloudformation Stack” is.

The “real code” is in setup-gitness-stack.ts where we create:

Required modifications?

You only need to set up “Secrets” in your repository with the names AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

After the initial deployment, you will need to modify the GITNESS_URL_BASE in https://github.com/Lock128/setup-gitness/blob/main/lib/setup-gitness-stack.ts#L104 to point it at the Load Balancer URL that has been set up for you.

Deployment

If you “fork” this repository and then push something to the “main” branch, a Github Action workflow will deploy this to your AWS account.

The deployed CloudFormation stack will contain everythign that you need:

The expected costs for this is roughly 50 USD / month.

Next steps from here

As mentioned above the code presented here is not production ready and it does not allow to use all of the functionalities of Gitness.

Things that you will need to consider / think of when using this code as a starting point:

  • What’s the backup schedule for the data on EFS?
  • What’s the required scaling policies?
  • Do you need a custom domain name?
  • Do we need to do security hardening – on the image, on the infrastructure, etc.?
  • Do we need to have multiple environments (development / production) for Gitness?
  • How to connect to this Git server from your VPC?
  • How to automate user creation or tie it to an existing AWS Identity Center?

As you can see, this is just a starting point of your journey to host your own Git Server on AWS 🙂

Please let me know if you have better ideas, suggestions or alternatives!

Views: 98