Dear AWS, how do I build & develop purely on AWS right now?

The announcements from AWS around deprecating certain services have raised a bunch of questions and concerns in the AWS community. 

As Jeff Barr wrote, these are the services:

S3 Select, CloudSearch, Cloud9, SimpleDB, Forecast, Data Pipeline, and CodeCommit.

This post will focus on Cloud9 and CodeCommit … and how I think this announcement impacts the “end to end” developer story for developers on AWS. We’ll also look at how the announcements impacts my “Go-To” services Amazon CodeCatalyst.

It is written from the perspective of a builder that mainly uses AWS tools for smaller side projects and can be seen as a “startup” that needs to quickly be up & running without much hassle.

Introduction

These annoucements, and the way that these deprecations where announced

Blog for CodeCommit

Blog for Cloud9

are in my humble opinion one of the worst possible ways. I know the teams at AWS have seen the feedback and I hope that there will be a clearer communication strategy going forward.

For me the combination of those posts with the assumption of CodeCatalyst being built on top of these services gives a very strange feeling on how much AWS is currently invested into Developers on AWS.

Let’s look at why I see a lot of impact of these announcements for builders and think about alternatives if you are using CodeCommit or Cloud9 for certain aspects today.

Tools required for SDLC

A few weeks ago I even dedicated a complete Shorts-Playlist on all of the Code* tools, looking at their usage and the approach to cover a full Software Development Lifecycle (SDLC) building on AWS.

In this series I drafted the diagram:

AWS Tools part of your SDLC until recent announcements

This being and end-to-end flow, AWS had at least two options to implement this process using their tools:

Either CodeCatalyst or a combination of different AWS Services

When CodeCatalyst was announced, I wrote about how CodeCatalyst can be used to cover all parts of your Secure Development Lifecycle (SDLC) process on AWS. Ever since then, there was an alternative on AWS using a combination of different building blocks: CodeCommit, CodeBuild, CodeDeploy, CodePipeline and others.

CodeCommit was a good, reliable managed Git server. For the purposes it solved, there weren’t many features to add. It was a managed service you didn’t need to think about and just “serve its purpose”.

Cloud9 was a hosted IDE, a development environment that users where able to access through their browser. This enabled builders to have a real IDE, even on old or underpowered computers, anywhere — even when being on vacation.

Developers on AWS are still able to use CodeCatalyst to cover for all parts of your product lifecycle or they had the alternative to use the different “building blocks” to compose their SDLC process. Both options gave value and helped AWS customers to solve certain aspects and problems.

Now, officially, only one option is left — CodeCatalyst.
CodeCatalyst is an integrated DevTools service that unites all of the building blocks under an opinionated, structured user interface. It was announced at re:Invent 2022 and went GA in early 2023. With the custom blueprints feature, it also enables builders to create project templates and share them with their team mates or dependend teams. Very powerful possibilities for teams to collaborate better and also share their best practices with other teams.

Those that didn’t need a “reliable managed Git server” where most probably using existing alternatives — that might solve the “job” better than CodeCommit — like Github, Gitlab or Atlassian. These users and AWS customers are not affected by the change.

What has changed with the July 2024 announcements — builders perspective

Now, the system landscape has changed.

Developers can not use Cloud9 anymore to develop software, they need to fallback to alternatives like Github Codespaces, Coder or Gitpod.

Developers cannot store their source code in CodeCommit anymore, they need to fallback to alternatives like Github, Gitlab or Bitbucket.

And given CodeCatalyst might be using CodeCommit under the hood and is using Cloud9 for the DevEnvironments – Can I really build something on top of CodeCatalyst going forward?

So this announcement of deprecation — without a “real” AWS native alternative — puts everyone building and developing software on AWS in the situation of needed to look for alternative setups.

Especially it forces you — if you are a small organization (or a startup) to engage with more than just one vendor as part of your SDLC lifecycle and process. I see this as a critical point to talk about aswell.

And, if you are building software or platforms on AWS where especially CodeCommit is part of application or the deployed architecture itself — You are now left without any option. If you want to integrate a Git server in your application on AWS, you will now need to self-host the git-server instead of using a managed service.

If you “just” needed a Git repository, quickly, fast and reliable — CodeCommit was the way to go. Now, you need to use a 3rd party alternative.

Now: What options on AWS do we have as builders?

What changed with the July 2024 announcements — business perspective

Looking at the announced changes from a different perspective, we need to acknowledge that AWS is a 90+ billion (90.000.000.000) dollar company. It is clear, being a business that aims to “make money”, that AWS needs to focus on services and solutions that are widely used, adapted and earn a good margin.

The reason might be that Cloud9 and CodeCommit where just not profitable enough to drive the expected growth of the business. Especially, as there are other services that do the same job better than Cloud9 and CodeCommit. So it might have been “just” a business decision to stop investing into these services and focus instead on Amazon Q that promises to help developers and builders on AWS.

This raises the question on which other services might soon or in the future be hit by exactly the same challenge. And – how is “success” measured by AWS on services? Is it “just” revenue or are there other points that are being considered?

But still — How this feels for me and questions I have (emotionally)

It feels like AWS has given up the game of engaging with their “Builders” and is now focused on the “Buyers” that “host” their applications on AWS.

If you think about how AWS started and if you look at how much effort AWS has spent this year on making us think that “Amazon Q Developer” is going to make our lives as developers easier…

How can I as an advocate for AWS as a platform be confident that I am valued as “Builder” on AWS? Will other services also disappear if they do not get enough traction?

And how much can I trust in Werner’s “Now, go build“?

How much “trust” can I put in the other Code* (CodeBuild, CodePipeline, …) tools on AWS?
With CodePipeline and CodeBuild getting a lot of notable updates right now (macOS, Github action runners, Stages rollback, …) the outsiders view is that at least these services are there to exist… but how much trust has the AWS team lost with Builders around the globe?

I’m eager to see how the different workshops, best practice documents, open source projects that use either CodeCommit oder Cloud9 (especially also AWS owned) will be adjusted and updated in the next weeks and months.

How much is also CodeCatalyst going to be the central place for Developers on AWS? How much updates will we see there?

How does this affect you – I would love to know!

I am really interested to hear how these announcements have affected your perspective on AWS and your view on the different AWS services.

Please share your thoughts either as a comment to this post or reach out to me personally!

What YOU can do next

You could now follow the advice from AWS and “migrate” away from CodeCommit or Cloud9 — but is this really what you want to do?
If you have a need to have a “Git server” or “Git repository” close to your applications on AWS, how do you do that?
You might need to host your own Git server on AWS….or you need to give up on that premise and fallback to alternative Git providers like Github, Gitlab, …

If you insist on having your own hosted Git within your AWS environment, there a few possible solutions…

…and potentially others that I am not aware of….

In order to host a “simple” Git setup I’ve recently made this repository public that deploys Gitness as a Git Repository on ECS. It will cost you roughly 50 USD/month. See also a relevant blog post.
Inspired by this, Jakub Wolynko did the same thing for Onedev – please see https://github.com/3sky/onedev-on-ecs if you would like to try that out.

As an alternative for Cloud9, you can use vscode.dev, which runs VS Code in the browser or other alternatives that are more integrated and personalized like gitpod.io or Github Codespaces.

But is this REALLY what you want to do if you are working on AWS only?

What I hope to get from the AWS team

As re:Invent is approaching fast and that usually sets the direction for a lot of AWS services, I really hope to get reliable information and roadmap clarifications around the AWS developer tools.

I’d like to understand if I can rely on CodeCatalyst, CodePipeline, CodeBuild, CodeArtifact, CodeDeploy, … and other AWS services that help developers to build software on AWS.

Does anyone know if this page ever mentioned CodeCatalyst? Please let me know!

In addition to that, I would love to get a better and more detailed overview on what the level of support will be that customers of the “deprecated” services will get: Security Updates? Priority Support?
Creating one page that summerizes that for all “deprecated” services would be amazing!

And – last but not least – make sure that Amazong Q knows which services you are deprecating!

Screenshot taken on 6th of September, 4pm CEST

If you’ve read this post until here, I would love to get your view and your feedback on this topic!

Thanks for the feedback I got before publishing this article and while I know you don’t agree with everything I wrote, it’s great to get your feedback, Monika, Raphael, Ran, Markus and others 🙂

Please let me know either in the comments or directly on my social channels — LinkedIn, X being the ones I still use mostly 😉 

Views: 634

A self-hosted CodeCommit alternative

A few weeks ago AWS CodeCommit became a deprecated service on AWS. This means, customers cannot create new repositories anymore – refer to this announcement for all details: Blog for CodeCommit

There are obviously a lot of alternatives to CodeCommit (Github, Gitlab, …) but if you need a “self-hosted” Git repository in your own AWS account this can become a little bit harder to provision.

If you’re looking for a “self-hosted” Git server, a bunch of tools come up:

  • Gitea
  • Gitness
  • Gitlab
  • …and obviously also others or just a “git” server running on EC2

As I wanted to be able to deploy something that works “out of the box” I looked at how to provision one of these alternatives on my own AWS account.

Gitness

Gitness is an open source development platform packed with the power of code hosting and automated DevOps pipelines. It’s biggest mantainer is Harness. It includes “way more” than just Git – eg Gitspaces, Pipelines, etc.

Deployment of Gitness

If you want to deploy Gitness, the docs point you at running it locally using Docker, or by deploying it on EC2 or k8s.

My aim was to “only” make the “Git” component available and because of that I’ve chosen Amazon ECS with EFS as storage to provision Gitness.

Open Source Code for the deployment of Gitness

I’ve set up a project on Github where all of the code examples are available for you to look at:

https://github.com/Lock128/setup-gitness

In the rest of the article, I’ll try to walk you through the most important parts of the project.

Please note that this code is not production-ready, it is more a PoC to showcase the direction that you could take.

Source Overview

We’re using AWS CDK and Typescript for Infrastructure As Code (IaC).

We have a “bin” directory where the main application is and a “lib” directory where the “Cloudformation Stack” is.

The “real code” is in setup-gitness-stack.ts where we create:

Required modifications?

You only need to set up “Secrets” in your repository with the names AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

After the initial deployment, you will need to modify the GITNESS_URL_BASE in https://github.com/Lock128/setup-gitness/blob/main/lib/setup-gitness-stack.ts#L104 to point it at the Load Balancer URL that has been set up for you.

Deployment

If you “fork” this repository and then push something to the “main” branch, a Github Action workflow will deploy this to your AWS account.

The deployed CloudFormation stack will contain everythign that you need:

The expected costs for this is roughly 50 USD / month.

Next steps from here

As mentioned above the code presented here is not production ready and it does not allow to use all of the functionalities of Gitness.

Things that you will need to consider / think of when using this code as a starting point:

  • What’s the backup schedule for the data on EFS?
  • What’s the required scaling policies?
  • Do you need a custom domain name?
  • Do we need to do security hardening – on the image, on the infrastructure, etc.?
  • Do we need to have multiple environments (development / production) for Gitness?
  • How to connect to this Git server from your VPC?
  • How to automate user creation or tie it to an existing AWS Identity Center?

As you can see, this is just a starting point of your journey to host your own Git Server on AWS 🙂

Please let me know if you have better ideas, suggestions or alternatives!

Views: 265

The state of CodeCatalyst in July 2024

I am personally using CodeCatalyst regularly for a lot of private projects, I also work a lot with other users of CodeCatalyst and I give feedback to the CodeCatalyst team regularly. In this post I look at the state of the tool in July 2024 and about how I make use of it on a regular basis.

A few more months in…

CodeCatalyst has been officially announced in december 2022 and reached GA in april 2023. Since then, it has been getting a lot of updates and changes, some of them you’ve potentially never had a look on.
In december 2023, major updates for enterprise customers were announced alongside other features like packages and Amazon Q integration functionalities.

CodeCatalyst Best New Updates in July 2024

Since last re:invent, CodeCatalyst has gradually increased the third party integrations with the option to have your source code stored in Gitlab, Github or Bitbucket. We have also seen the expansion of Custom Blueprints to CodeGeneration for repositories stored outside CodeCatalyst itself.

Just recently, we have also seen the possibility to have more than one space attached to a single IAM identity center, which allows further usage of CodeCatalyst for more enterprise customers.

CodeCatalyst also announced the possibility to expand packages usage to other providers than just npm – you are now also able to store maven based artifacts or OCI based images in packages.

Major updates to custom blueprints and additional blueprints allow and anble you to on the onside import source code into CodeCatalyst and on the other side to create a custom blueprint out of an existing project. This should make creating blueprints more accessible.

For a few months it has also been possible to include “approval gates” in CodeCatalyst workflows. This is a very limited functionality, but it still allows some important use cases.

Is CodeCatalyst ready for prime time?

It still depends.

While CodeCatalyst has drastically improved and matured over the last 12 months, there are still a few things that need to get better before I would 100% recommend you to use it.

Things that mainly concern me as of now: CI/CD capabilities and integration with AWS services.

The CI/CD capabilities are still limited and need to be improved to be more flexible and integrated. Approval rules need to be more sophisticated and allow some more specification.

If you already have CI/CD workflows or branch permissions set up in a tool of your choice, having “import” functionalities that translate existing Github Actions, Jenkins pipelines or Gitlab workflows into CodeCatalyst workflows is missing as well as the option to automatically set up branch permissions.

Other than that, CodeCatalyst is pretty much ready to be used for prime time and it has some functionalities that are outstanding and should be marketted more.

Next steps? What I think could come next…

The brave option

I still believe that that most underrated functionalitiy of CodeCatalyst is the Custom Blueprints functionality. If you’re living in a k8s world, Backstage has been leading, together with others, the field of “Internal Developer Portals” that empower developers perform actions quicker and more eficient in their day to day live. Especially Backstage starts with the possibility of scaffolding projects and generating code. However, Backstage does not allow you to keep track of changes to the relevant templates later.

Custom Blueprints – and also “existing Blueprints” do empower developers to do exactly the same thing.

Given CodeCatalyst has already been opening itself with other third party integrations like allowing a full Github, Gitlab and Bitbucket integration, I can see a potential of opening CodeCatalyst up even further.

With the given and already available marketplace in CodeCatalyst – that is not yet used very much – this could be opened up to allow other providers to add additional integrations, actions, blue prints.

Still the team would need to add additional functionalities like dashboards, widgets, … to make CodeCatalyst like an “Internal Developer Portal”.

What is unclear to me is whether AWS will be brave enough to perform another 1-2 years of investment into CodeCatalyst before it can become the central place for developers on AWS. I am also not sure AWS will finally go All-In on CodeCatalyst or if they will continue to invest into the existing Code* tools (CodeCommit/CodePipeline/CodeBuild/CodeArtifact).

The usual way for AWS developer tools

AWS will continue to invest half-focused and try to stay “on track” to help a huge customer base to achieve the simple things with CodeCatalyst. Integrations to other AWS services will be missed, the adoption rate will be small. With this kind of investment, AWS will have multiple solutions of Developer Tools (CodeCatalyst vs. CodeCommit/CodePipeline/CodeBuild/CodeArtifact) in the portfolio that both do not solve “all” problems and usecases but serve different customer bases.

What I think will happen

Give CodeCatalyst is build in different service teams we will see some teams heavily investing into making “their” part of the product successful (e.g. “Packages”, “CI/CD” or “Amazon Q in CodeCatalyst”). We will start seing these unique capabilities reach other parts of AWS services or potentially also other platforms. CodeCatalyst as a product will continue to exist but the different service teams will start to focus on where they can make more “money”. CodeCatalyst will not be able to deliver the promise it had when it was announced as the “central place for DevOps teams on AWS”. CodeCatalyst functionalities will be made available through the AWS console. With that, CodeCatalyst as “the product” that I was hoping for will cease to exist.

What do you think about my ideas and assumptions? Do you think I am wrong?

Drop me a comment or a note, I’d love to hear what your take on the future of CodeCatalyst is!

Views: 703

Another year in the community – Thank you, AWS community team #thankfulforest2024 #firevalleyrocks

The year 2023 is close to its end and we’re approaching “Holiday Season” – which is one more reason to take a few minutes to say THANKS to the ones that work every single day to empower the AWS Community.

We did this last year, too – so it was about time to try something else – and the community did it again: All the trees of the Community forest

Saying “thanks” with my speciality service

I’ve found a way to make a tree shine in CodeCatalyst using the Workflows – its not as colorful as that Jenn did and not as detailed as Brian’s approach … and it should definately not try to replicate the team structure or org chart, but it shows that all of the work that the AWS Community team does, all of the support, guidance and investments make the AWS Community a strong foundation of everyone that wants to be part of it!

The community is open for everyone, you can even start your own Meetup easily.

Thank you, AWS Community Team

I am really thankful to be part of the AWS Community and it’s energizing to see the ideas, the sessions, the discussions that we all have together. You, Ross & team, make this possible every single day. Thank you for empowering us, for guiding us and for enabling us to be successful.

Here’s the code for my CodeCatalyst workflow:

Name: firevalleyrocks
SchemaVersion: "1.0"
Triggers:
  - Type: Push
    Branches:
      - main
Actions:
  Ross:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Ross!"
        - Run: echo "Thanks for all of your support in 2023!"
  Taylor:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Taylor!"
        - Run: echo "Thank you for making the Heroes a true community!"
    Compute:
      Type: Lambda
    DependsOn:
      - Ross
  Jason:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Jason!"
        - Run: echo "Thank you for making me start my Community Journey and for making the Community Builders what they are!"
        - Run: echo "Sorry, but you're red!"
        - Run: xxx
    Compute:
      Type: Lambda
    DependsOn:
      - Ross
  Maria:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Maria!"
    Compute:
      Type: Lambda
    DependsOn:
      - Ross
  Ernesto:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Ernesto!"
    Compute:
      Type: Lambda
    DependsOn:
      - Taylor
  Farrah:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Farrah!"
    Compute:
      Type: Lambda
    DependsOn:
      - Taylor
  Lily:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Lily!"
    Compute:
      Type: Lambda
    DependsOn:
      - Jason
  Thembile:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Thembile!"
    Compute:
      Type: Lambda
    DependsOn:
      - Maria
  Susan:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Susan!"
    Compute:
      Type: Lambda
    DependsOn:
      - Maria
  Albert:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Albert!"
    Compute:
      Type: Lambda
    DependsOn:
      - Ernesto
  Shafraz:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Shafraz!"
    Compute:
      Type: Lambda
    DependsOn:
      - Ernesto
  Wesley:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Wesley!"
    Compute:
      Type: Lambda
    DependsOn:
      - Lily
  Ben:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Ben!"
    Compute:
      Type: Lambda
    DependsOn:
      - Lily
  Will:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Will!"
    Compute:
      Type: Lambda
    DependsOn:
      - Susan
  Nelly:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Nelly!"
    Compute:
      Type: Lambda
    DependsOn:
      - Susan
  Community:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Community!"
    Compute:
      Type: Lambda
    DependsOn:
      - Nelly
      - Will
      - Ben
      - Wesley
      - Shafraz
      - Albert
  COmmunity:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Community!"
    Compute:
      Type: Lambda
    DependsOn:
      - Community
  Commonity:
    Identifier: aws/build@v1.0.0
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: echo "Hello, Community!"
    Compute:
      Type: Lambda
    DependsOn:
      - COmmunity

CodeCatalyst at re:Invent 2023, Youtube and a Speakers Directory

In 2023 I’ve become lucky. I’ve started my own YouTube channel, where I present all of the release highlights of re:invent 2023 for CodeCatalyst, I’ve become an AWS Hero but more important than that, I’ve made a lot of friends around the globe. I’ve empowered others to become part of the community and I’ve challenged others with questions, tasks and ideas like the Speakers Direcory.

Thank you for making my year 2023 unforgettable and for making me smile when I think about what we achieved together!

Views: 6731

Application Composer levels up a lot and adds amazing IDE integration capabilities

In this post we’re going to look at the new functionalities that have been added to Application Composer

In this post we’re going to look at the new functionalities that have been added to Application Composer by re:Invent 2023. After announcing the support of all CloudFormation resources earlier in the year, Application Composer now allows editing StepFunctions within the same user interface and – even cooler – announces the integration of an IDE plugin that allows developers to build serverless functions locally.

Application Composer as a serverless, rapid prototyping service adds additional capabilities to empower developers building serverless applications

Application Composer, that was originally announced last year at re:Invent 2022, has gotten a lot of major improvements thoughout 2023. As we are right at re:Invent 2023, its time to look back on which new capabilities have been added and how they influence building serverless applications using AppComposer.

Supporting all CloudFormation resouces

Already a few weeks ago the team announced that all over 1000 CloudFormation resources are now supported by AppComposer. This really gave a big update and make it simpler to build all kind of serverless applications. However, as this only alled AppComposer to expose the resources, this still requires the developer to know all required connections between the different resources. I personally would love to get more “supported” resources (just like L2 resources in CDK) to be made available as part of AppComposer. I would hope that this will be an additional functionality soon.

Integrating additional services

With the integration of the Stepfunctions Workflow Studio within the same interface, developers can now build and end to end application within composer before using the generated SAM or CDK templates to trigger the deployment. As a next step I think it would be great to also be able to define Eventbridge Rules & Pipes within the same interface.

Local development and IDE integration

AppComposer announced a Visual Studio Code integration that makes it possible to build and design serverless applications right from your IDE!

With this feature, you can visualize your serverless applications without being within your browser or the AWS console – start building, whereever you are and whenever you want!

I have not been able to try out this functionality yet, but especially the integration with sam connect that allows to also directly deploy the changes you made to your picture / template will make a big different in building applications using AppComposer.

Also I think we should not underestimate the possibility that this offers to vizualize existing CloudFormation templates through either the IDE plugin or the AWS Console. This will help to explain big and difficult already existing applications and empowers teams to have a fruitful conversation about changes they would like to implement in existing templates, as have a visualization makes the conversation easier.

What’s next for Application Composer? What are my wishes?

Already last year I have asked to integrate AppComposer in CodeCatalyst and I believe that this would be an awesome possibility to quickly start serverless projects. Application Composer today feels like a playground – to make the service more usable, it needs to have a “deployment” component that allows you to automate the lifecycle of your serverless application (including a full CI/CD pipeline).

I also last year asked for creation of CDK out of Application Composer – or even importing it – but instead of investing into that direction AWS recently announced the existance of the CDK Builder Tool – wouldn’t it be better to merge those initiatives together?

As already mentioned above, supporting additional “CDK-L2-like” patterns – or maybe the “Patterns” from serverlessland.com would be amazing – so users do not need to know how to set up IAM roles, connections between API Gateway and Lambda, … manually would make this a much more usable product!

What are your thoughts around the recent announcements of AppComposer? What are your experiences with it?

Views: 422

re:Capping re:Invent 2023 – Not everything that happens in Vegas should stay there! Let’s go and build!

In this article I will try to re:Cap a few of the announcements at re:Invent 2023 but also share my personal experiences and learnings that covers what I think that should be shared with the world…!

What happens in Vegas…

…should not always stay in Las Vegas! This year’s re:Invent has been another great experience for me and it was amazing to meet AWS enthutiasts from all of the world. I’ve learned a tons of stuff, saw a bunch of cool sessions and also experienced to be part of a big family. All the friendships that have been build in the past few days, the shared knowledge and experiences that have been shared have a big influecnce on myself and shape me.

The technical aspects of re:Invent

This year the technical aspects of re:Invent existed but where not as important to me as they used to be in my previous attendances. Of course AWS hd a bunch of important announcements – some of them bigger, some smaller. Renato has them written up at InfoQ and the AWS Newsblog has them covered too. Luc, the winner of this years “Now, Go Build” award 2023 has created a web application that helps you to read all of them and not miss a single one.

For me, there are a few that stand out:

Of course, there where a bunch of other announcement, minor and smaller ones, but these are the ones that I have remembered and thus they are meaningful to me. Now let’s move over to the more important aspects of re:Invent!

The community aspects of re:Invent

re:Invent 2023 has been once more a gathering of the AWS Community in one place and it has brought a lot of us together to talk, laugh and align. Not everyone was able to join us due to different reasons – but I am sure that you have felt the power of the community throughtout the week by following us on the different social channels.

Being part of the AWS Heros

As I posted last year, going to re:Invent means meeting with friends and getting together. Being an AWS Hero, made it more intense than before: We feel community out of our heart and that’s what makes us strong. Wherever I was in Las Vegas, I saw a fellow Hero.

We all have super powers and our powers are different. One of my super-powers is connecting people – and I hope that I was able to show this in the last few days.

Others have other powers – a few of use were able to present one of their talks – Anahit with her spciality around MSK, Anurag around data patterns and Ran on Lambda Power Tools. Others are great listeners and others have the vision of how things need to or should look in a few years – it was great to see everyones powers in one place and I know that combining them we can incluence to make things better!

Thank you, Taylor and the rest of the team, for creating this group and bringing us together again!

Working with Builders, User Group leaders and others from the community

The AWS Community consists of so much more than the Heros. Thank you, dear Community Builders – lead my Jason and the team – for being an unbelievable source of power throughtout the week. Your entuthiasm, your great ideas and your dedication are what makes us stronger. I’ve been reading a lot of the posts from Builders around the globe that were not able to make it to Las Vegas and it is energizing to see that.

The User Group Leaders that we have world wide on the other side help to thrive the AWS Community across the whole yearand bring us together regularly — to learn, to play or to share knowledge. Thank you all, for helping us to shape where the community goes and for making the community successful. I was glad to be able to meet a lot of you and share my experiences as welll as listening to your experiences.

I had the great pleasure to get the whole team of core contributors of the Speakers Directory together and we were able to present our project as well as take a picture of all of use 🙂

We are going to continue our investment and will help user group leaders to find speakers through our tool!

Working with AWS employees

This year, I’ve joined the club of many other Heros that go to sessions where they can meet AWS service team members that they have worked with before 🙂

I attended a few CodeCatalyst sessions to meet the team that I’ve been working with for more than 12 months “live and in person” and loved to see the energy and innovation live on stage – but I also attended other sessions just to say HI to certain speakers.

Employees at AWS are smart and can often tell you the perspective of WHY something has been build and it’s great to know some more background of a new feature. Thank you all for spending time with me and sharing your thoughts and passion with me!

To those AWS employees in the community and DevRel team – another big THANKS for making the event unforgettable with all of your dedication and support – I love spending time with you and creating new ideas on how to make the AWS community stronger and more engaging than ever before!

A look ahead…

As I try to use my time on the flight to put my head around what I am taking away from the last few days and from re:Invent 2023, I’m still digesting, as many others, what we have all learned and heard.

A few key take-aways:

  • AWS doesn’t feel “secure” anymore to be a market leader
  • innovation at AWS is coming (Q), but it’s still early stages
  • AWS keeps listening to their customers (see the DB2 RDS announcement and the StepFunction HTTP Integration)
  • Community Sessions (COM or DEV track) are the ones to attend at re:invent, or sessions that are AWS + a customer (level 300/400)

What I’m considering to do in the next 3 months

First of all I’m planning to cover the CodeCatalyst announcements at my YouTube channel to explain the impact of the new features to interested enterprise customers.

I’m also looking at trying out a lot of the cool things that have been announced in our AWS Speakers Directory Project, besides hosting multiple User Group meetups of the AWS UserGroup Bergstrasse.

What I’m considering to do in 2024

Of course I will continue my engagement in the AWS Förderverein DACH – we are planning another AWS Community Day in Munich next year!

I also plan to continue my work with the the CodeCatalyst team to shape the product – please let me know if YOU have input on what thte next important steps are.

I would love to work with the AWS team to, for 2024 at re:Invent organize another pre:Invent Community Hike and to talk about the possibility of hosting a complete track at re:Invent where Community Members join forces with AWS employees. I listened to a session (Ran Isenberg and Heitor Lessa) and that was a very powerful message.

Last but not least, I would like to help community members to grow and shape their careers in Cloud – if you need help or have questions, do not hesitate to reach out for questions, I’m happy to help or to connect you with someone that can help!

Thank you for reading until this point, if you have any feedback, let me know!

Views: 259

Amazon CodeCatalyst’s packages support – a glimpse at what’s to come for artifact management

In this post you will get a short introduction and my personal assessment on how the new Packages component in CodeCatalyst can help you to set up your complete SDLC in Amazon CodeCatalyst.

Only npm supported – a early launch to show what we can expect going forward and to get feedback from users?

With the initial launch Amazon CodeCatalyst allows to manage npm package repositories inside CodeCatalyst. When seing this for the first time I was a bit concerned that this decision – to initially launch with only npm support – will not help a lot, as I expect that organizations would need to store other type of artifacts (jar files, containers, python paackages, …). But I think that at least for Cloud Native Projects that use typescript this will solve the problem of storing artifacts and accessing them natively within CodeCatalyst. Let’s look at how they can be used today.

Using packages repositories in CodeCatalyst

Today you can set up package repositories in CodeCatalyst and connect to upstream (public or private) repositories. You can also change the sort order and the upstream order of the package repositories. The documentation covers the different possibilities to set up repositories very well. You can access the package repositories from your local machine by setting up the connection locally

npm config set registry https://packages.region.codecatalyst.aws/npm/space-name/proj-name/repo-name/

After setting up multiple repositories you can set the order of retrieval and usage. You can also set it up as a “pass-through” repository and allow access to public repositories.

Using packages in CodeCatalyst

Packages in CodeCatalyst are integrated with workflows. You can read or store packages with native actions or by setting up the npm repository manually.

You can also read and publish packages from other systems by using Personal Access Tokens (PAT). The documentation outlines currently supported client commands.

The CodeCatalyst workflows will by default use the set up repositories that you have set up within CodeCatalyst.

What I think we need next in packages

With the launch of the packages component in CodeCatalyst AWS has added a definately missing functionality in CodeCatalyst. Users are not able to store artifacts and share them within a space which makes it possible to create, re-use and deploy immutable artifacts natively within the tool.

This is a much needed functionality, but as I already mentioned, I believe that limiting this to npm packages only limits the use cases for it. I would have expected npm, python and docker support at launch and was a bit disappointed that this was not included.

I do also believe that the “packages” functionality should be better integrated and allow further configuration options, especially when loading/reading packages from upstream repositories – e.g. being able to limit to only certain licences or ensuring that packages included are still “supported” (and regularly updated) or security-scanned.

These type of functionalities would have made the new option “meaningful” and would have empowered developers to build better software too.

I can imagine that we will see python, java and dockersupport pretty soon due to these reasons and it would be great to also support “internal” repositories as “upstream” repositories.

How do YOU think that this new functionality will help you to adapt CodeCatalyst?
Please drop me a message on socials or an E-Mail!

Views: 552

Amazon CodeCatalyst’s integration with the IAM identity center allows SSO with your own IdP

In this post you will learn how the new integration of the IAM Identity center with Amazon CodeCatalyst allows to use your own IdP as single source of truth of users for CodeCatalyst spaces. We will look at how you can connect a space to an IdP and on how this can be used to set up certain permissions based on your IdP roles.

Single Sign On (SSO) using IAM Identity Center empowers CodeCatalyst users to use their own IdP to provision user accounts in CodeCatalyst

With the new integration for Amazon CodeCatalyst into the AWS Identity Center organizations can now connect to their own existing IdP (AzureAD, Ping, Okta, …) going through the AWS Identity Center.

Benefits of the new integration

With this new integration users and organizations go to the Space settings in CodeCatalyst and choose to integrate with an existing instance of the IAM Identity Center.

This also connects potentialy to any other IdP that uses the IAM Itentity Center as a proxy. Administrators will only need to manage roles and permissions in on central place. The roles from the IdP can then be mapped to certain team roles in Amazon CodeCatalyst.

What I don’ t like about the new integration

There are two things that I didn’t like when looking at this integration:

Why do we need to go through the IAM Identity Center and cannot allow direct SAML or OIDC integration for a specific space? With the “need” to have an IAM Identity center organizations also need to directly connect to an existing AWS Account to manage connectivity – and this once again brings back a “tooling” account that initially I was hoping to replace with CodeCatalyst.

The second thing that I missed was the possibility to automatically map existing roles to groups in Amazon CodeCatalyst – today this is a – one time – manually effort that needs to be conducted. If you need to do that only once for your organization that isn’t a problem, but if you decide to have multiple spaces for your organization and need to perform this multiple times, then this can be very time consuming.

Summary

I believe that this change will have a very big impact on the adoption for CodeCatalyst as administrators and security teams in organizations will make it easier to allow the usage now that there is a central place to manage identities. And with the possibility to actually try out CodeCatalyst in an enterprise world, I hope that we will also see more adoption of CodeCatalyst and with that also a more streamlined and better communicated roadmap of CodeCatalyst.
If you would like to learn more on how to activate this, please refer to the documentation.

Views: 178

Custom BluePrints in CodeCatalyst – templated projects that empower you to build better software

In this post you will learn how to build a Custom BluePrints in Amazon CodeCatalyst. You will get to know how Custom BluePrints allow you to generate a consistent setup of your projects and CI/CD workflows and how you can leverage this to empower your teams to be compliant with rules and deployment best practices. AWS has announced this today at his annual user conference AWS re:invent 2023 in Las Vegas.

Custom BluePrints – what they do, why they are there and how you can use them

Custom BluePrints can be seen as “project templates” that you can build and offer to all of your users of a space. As a user of CodeCatalyst you should already be aware of the general concept of a blueprint: It’s a templated project that you can start your software project on.

Custom BluePrints take this concept to a new level: Now you can build BluePrints for your projects!
And that’s not it: a project can be depended/generated by multiple BluePrints and a project generated from BluePrints can also become a BluePrint! yey!
You do not start from zero and you are now also abled to add a BluePrint to an existing project.

Think about what this means for you and your organization: You can set up certain default workflows, permissions, dependencies, environments, etc. and apply that to all of your new projects by default, without the need to “touch” anything.

And that’s not “it” – there is more: CodeCatalyst also allows you to “update” projects that have been build from a Custom BluePrint!

Think about the scenerio where you ahve over 20 microservices that all follow the same CI/CD pattern, including governance rules and dependencies. They all have the same dependencies and pre-requisites (e.g. AWS account permissions, environment setups, etc.). Now one of the dependencies has a critical, security relevant defect and you urgently need to upgrade all of these projects…

CodeCatalyst Custom BluePrints enables you to do exactly that with just a few clicks and pull request approvals!

I hope you are curious to look at this new functionality in details now, keep reading!

The technology behind blue prints – the power of Projen and Code Generation

Custom Blue Prints are powered by Projen under the hood – to learn more about Projen, please look here. Similar to a projen.rc.ts you will need to create a blueprint.ts file in the src folder of your BluePrint project. There you can then define the rules and automations which will be applied when the BluePrint is used as a starting point for a new project. Currently the Custom BluePrint SDK allows you to define Wizard configurations, Workflows, Environments, Source Repositories, Pre-Configured Dev Environments, … Using Projen as an underying technology the team is able to re-generate the code for your project and create a pull request on your behalf for the source repository of your project i there is changes. And this is great!

Building your first BluePrint

Building your first BluePrint is easy!

Create your BluePrint project

It starts with creating a new Custom BluePrint project in the CodeCatalyst UI using the BluePrint builder. This one can be accessed from the Space “Settings” page. After going through the wizard, this will create a new project for you in CodeCatalyst. In this project, you can now either checkout the source repository into your local IDE or you can use the DevEnvironments to directly get you started to work on the project.

Building and Editing Custom BluePrints can be a HIGH RISK

As you are reading this and as the functionality has only just been announced, it might be early – but it is never too early to warn about this:

If you edit a Custom BluePrint and potentially introduce security problems – maybe even unsecure dependencies – you might expose everyone that uses your BluePrint to challenges.

Better set up additional protections for all of your BluePrint projects!

Edit the sources of your BluePrint Project

Now that you your project is set up, you can start building your own components and parts of your project.

Be cautious to edit package.json in your BluePrint project – you can, but it might break some of the integrations. The typescript project is set up for you to be able to preview and publish your BluePrint.

The main “sources” of your BluePrint and the definitions will be in the src/blueprint.ts file. Initially, the project comes with a simple wizard set up with only a few parameters. It copies only the contents of the static-assets directory when being executed.

I’ve not been able yet to try out a lot of the functionalities and possibilities of the SDK, but still I was able to create a custom BluePrint that can be used to deploy a Flutter Web application with a Serverless Backend.

What I found out is that it will be a huge effort to set up the BluePrint in combination with a Wizard. This is not a 4 hour task, we’re rather talking about a week to get started. Further details on this – see the “Testing” section. This might also explain why this functionality is part of the new Enterprise tier for Amazon CodeCatalyst.

Please read the available options of package.json, but to get you started: use npm run blueprint:synth or yarn blueprint:synth to generate the BluePrint locally.

This will also execute any local unit tests that you have build for your BluePrint.

Testing your Blue Print locally

Using npm run blueprint:synth or yarn blueprint:synth will generate the BluePrint bundle locally and execute unit tests. It will generate a version of your project created by the BluePrint in the folder synth/synth.[options-name]/proposed-bundle/. This will use the default options you provided.

It is also possible to simulate the wizard adding the --cache parameter.

You could also build Unit Tests or SnapShots tests and integrate them into this lifecycle step of the build procedure.

Publishing your BluePrint to the Space Catalog

Use npm run blueprint:preview or yarn blueprint:preview to upload a “preview” version of your BluePrint to your CodeCatalyst space. This will make the Custom BluePrint available but not make it available in the catalog yet, so users cannot use it.

It also automatically increases the version number for your BluePrint.

Testing your Blue Print through the UI

After publishing your BluePrint preview from your local environment, you will now see this preview version within your Space Settings. From there, you are able to create a project out of the Preview version of your BluePrint.

Here you will need to test all options of the wizard and in addition to that verify that the customizations that you added have been correctly generated after you went through the wizard.

I have made the experience that a whole testing cycle can take 3-5 minutes – so I really recommend you to look at local unit tests.

After you have finished your tests, you can add a specific version of your BluePrint to the BluePrint catalog. This is currenty only be possible through the UI in the Space Settings.

Creating (one or many) projects from your BluePrint

Now you are ready to start creating projects from your BluePrint!

Go to the normal “Create Project” screen – after using “Create from Blueprint” you will now have the option sect “Space Blueprints” which is where you will find your newly created BluePrint.

This should now generate your project from your BluePrint and the structure should be as you wanted it to be. 🙂

You can also add additional BluePrints to an existing project – this allows you to incorporate multiple BluePrints in a single project, a combination of multiple “default” project settings.

Please be aware that this will remove manually added or created resources that are part of the BluePrint that you are adding – it will overwrite manual changes that “interfere” with the BluePrints code.

Upgrading your BluePrint

Upgrading your works similar to creating it – upload a new “Preview” version, add the new version to the catalog.

Upgrading your projects to a new BluePrint version

You can currently update the used Blue Print vesion for a project by going to the project settings, to the Blue Prints section there and then by loading up one BluePrint. Choosing to upgrade to a different or newer version will generate the proposed changes and make them visible to you before you apply them.

What’s next for Custom BluePrints?

Oh wow, this is one of the new features that I am most excited about for CodeCatalyst! There is so much potential! I have to explore further the possibility to have multiple BluePrints attached to a project, the possibility to remove a BluePrint from a project, and all of the other things that I didn’t touch yet.

Here’s stuff that I would love to get for Custom BluePrints:

  • auto-apply updates to ALL projects that are from the same BluePrint automatically
  • share Custom BluePrints with other Spaces
  • a Custom BluePrints market place where developers (like me) can put “their” BluePrints in and organizations can “buy” a BluePrint from

I also believe that the team needs to work a bit more on the developer experience when building Custom BluePrints, e.g. by pre-suggesting automated integrated tests (integration or snapshot tests). Publishing a new version to the catalog should also visualize the changes that are going to be added to other projects before you actually publish the new version.

I am a bit disappointed that the “free tier” does not give you the possibility to try out this functionality by being able to add one or two Custom Blue Prints to your space, but maybe this is something that will be possible as soon as the Blueprint SDK is stable.

What do you think of Custom BluePrints? What are your main wishes? Please reach out to me and let me know!

Views: 1217

Hey, I’m going to San Francisco and Vegas to meet friends at re:Invent 2023

In this post I’m writing about how getting to re:Invent 2023 was and about how the last year has made a big impact on my personality and my efforts in the AWS community.

Thinking it couldn’t get better last year, 2023 changed me once more and leveled me up

While going to re:Invent 2022 I wrote down my thoughts and went home from re:Invent expecting things could not get better for me. But this year has prooven my expectations wrong, as the last 10 months have been life-changing in a lot of different aspects.

Blogging & Starting a YouTube channel – winning 1.5 hackathons and starting a project

When I reached my home town after last years re:Invent I knew that this community is what makes me happy and gives me back so much that I also would like to give something back. And so, I formed a few ideas in my head and started a few initiatives that I would like to share with you today:

Blogging & Writing

I have been blogging both on dev.to and on my personal blog – writing about topics and things that I do or did on a regular basis, trying to share experiences of day-to-day things and building experiences.

The most important thing for me he is to always be transarent and authentic – speaking out on things that I liked and challenges that I’ve faced.

For you reading this – thank you, for being part of my journey, for giving me feedback and for being you! Please let me know if you have any feedback on what I should change or do better.

Kicking off a YouTube channel

The last three years of my career have been focused on building secure and reliable CI/CD pipelines! But hey, there is so much more to learn. As part of my travel back to my home town after re:Invent, I had the thought to share my experiences and the experiences of other builders on a “podcast-like” YouTube channel which is today known as @cicdonaws.

Learning on how to produce a YouTube video to how to set up a good lightning and sound, preparing a script and creating better thumbnails – a lot of learnings, experiences and hours went into the videos that I have been producing.

This only kicked off because of all of the great builders that joined me on the show and shared their projects, learnings and experiences with me. Thank you all, every single one, for your time, dedication and experiences. Keep up sharing your knowledge, as this is how we can all grow!

Winning 1.5 hackathons and making a small project something big

Back in march Lily announced a, Community Builders-internal hackathon where it was possible to win some prices – and I thought I had a great idea on what to build that actually fit into the “rules” of the hackathon. And this is how the project started that has since then used up countless of hours of my after-work evenings and weekends. Millions of Slack messages have gone from Germany to US, from US and Germany to Africa and back – we kicked of a real project with our Speakers Directory that is finally ready for prime time! After becoming second in the first Hackathon, we also submitted the same project with a few tweaks for a 2nd hackathon powered by Hashnode and Amplify and this also brought us “honourable mentions” in the list of top 10.

We’re ready now to get more of you to add your talks and events! Please sign up and give us feedback, we are very proud of what we have produced so far and eager to get all of you on board!

Thanks to Danielle, Julian, Matt, Raphal, Baimam for actively working with me on this fun project – and thanks to everyone else that has supported us through 2023 to make this a bigger thing!

Organizing a community day – the EU (more technical) edition of re:Invent

Another piece last year has been organizing the AWS Community Day DACH that happened in Munich this year – it was fascinating to be part of the team that brought more than 500 AWS nerds together, including over 25 AWS Heroes and 30 Community Builers!

Thank you all for attending – and thanks for the sponsors for making this possible!

We’re up for doing this again next year – further details to be announced soon! Looking forward to see you there!

Hey, hiking before re:Invent is what you need!

And then, there is this “secret thing”: Ever since I went to re:Invent I used the sunday before re:Invent to go out “hiking” somewhere close to Las Vegas. The first two visits together with colleagues were three persons – a pretty cool small group.

Last year, they did not go with me to re:Invent, so I asked Community Builders to join me – at the end, we were nine persons from I think 8 different countries hiking together from 10 am till 5 pm – we had a lot of fun, so I decided: Let’s do this again!

Somehow, this became a bigger thing – for this years hike, we have over 50 Community Builders, Heros and UG Leaders signed up. The group became so big that the AWS Community team is helping us this year by sponsoring the transportation.

Thank you all, that you are supporting this thing – and thanks to everyone participating, I am sure this will be an amazing experience for everyone!

How and Why what I do for the community changed

Since June I can name myself an AWS Devtools Hero – which honors me a lot but also drove a bit on how and what I do for the AWS community. My work has become a little bit less “public” – a lot of my hours that I have “free” went into talking to AWS Service Teams, into mentoring other builders and helping them grow. This has lead to less new videos on YouTube and less blog posts – but instead, I’ve helped others to grow and to invest into the community – supporting the “re-start” of the AWS UG Frankfurt, the re-start of the User Groups Mainz, Karlsruhe and Heilbronn and overall making other builders “grow”. This makes me happy. Thank you all, that you are part of my journey.

Making new friends

This year has also shown to me how important relationships are and how much you can make friends by supporting each other. Friends, I’ve never met in person (Matt), friends I’ve finally met in September (Ran) and others like Markus, Thorsten, Philipp and Raphael that are part of most of my days and that I write 100s of Slack messages per week.

This is what community means! Making friends, learning, growing. Thank you, for being you!

What’s next for me?

Changing my role at work

I’ve recently moved to the global Platform Architecture team here at FICO and that gives me a lot of new possibility and learning opportunities.

The platform that FICO is building is an important one and helps organizations around the globe to take better decisions – I’m proud to be part of the team and looking forward to make this bigger than it is today.

Community Day DACH 2024

We’re already planning next year. Stay tuned for some announcements…soon!

Supporting and mentoring builders around the globe

I’m up for helping other builders grow – if you’re intersted to collaborate with me or to learn from me, reach out to me and we can talk!

If you have any CI/CD or CodeCatalyst specific topics that you would like to talk about on my channel – reach out to me!

If you need feedback, help or advice on anything (e.g. a blog post or anything else) – let me know and reach out!

Looking forward to an amazing re:Invent 2023 with a lot of friends in Las Vegas!

Have a great week – don’t be a stranger and say HI if you see me around!

Views: 416