Category: CodeCatalyst

Amazon CodeCatalyst’s packages support – a glimpse at what’s to come for artifact management

In this post you will get a short introduction and my personal assessment on how the new Packages component in CodeCatalyst can help you to set up your complete SDLC in Amazon CodeCatalyst.

Only npm supported – a early launch to show what we can expect going forward and to get feedback from users?

With the initial launch Amazon CodeCatalyst allows to manage npm package repositories inside CodeCatalyst. When seing this for the first time I was a bit concerned that this decision – to initially launch with only npm support – will not help a lot, as I expect that organizations would need to store other type of artifacts (jar files, containers, python paackages, …). But I think that at least for Cloud Native Projects that use typescript this will solve the problem of storing artifacts and accessing them natively within CodeCatalyst. Let’s look at how they can be used today.

Using packages repositories in CodeCatalyst

Today you can set up package repositories in CodeCatalyst and connect to upstream (public or private) repositories. You can also change the sort order and the upstream order of the package repositories. The documentation covers the different possibilities to set up repositories very well. You can access the package repositories from your local machine by setting up the connection locally

npm config set registry https://packages.region.codecatalyst.aws/npm/space-name/proj-name/repo-name/

After setting up multiple repositories you can set the order of retrieval and usage. You can also set it up as a “pass-through” repository and allow access to public repositories.

Using packages in CodeCatalyst

Packages in CodeCatalyst are integrated with workflows. You can read or store packages with native actions or by setting up the npm repository manually.

You can also read and publish packages from other systems by using Personal Access Tokens (PAT). The documentation outlines currently supported client commands.

The CodeCatalyst workflows will by default use the set up repositories that you have set up within CodeCatalyst.

What I think we need next in packages

With the launch of the packages component in CodeCatalyst AWS has added a definately missing functionality in CodeCatalyst. Users are not able to store artifacts and share them within a space which makes it possible to create, re-use and deploy immutable artifacts natively within the tool.

This is a much needed functionality, but as I already mentioned, I believe that limiting this to npm packages only limits the use cases for it. I would have expected npm, python and docker support at launch and was a bit disappointed that this was not included.

I do also believe that the “packages” functionality should be better integrated and allow further configuration options, especially when loading/reading packages from upstream repositories – e.g. being able to limit to only certain licences or ensuring that packages included are still “supported” (and regularly updated) or security-scanned.

These type of functionalities would have made the new option “meaningful” and would have empowered developers to build better software too.

I can imagine that we will see python, java and dockersupport pretty soon due to these reasons and it would be great to also support “internal” repositories as “upstream” repositories.

How do YOU think that this new functionality will help you to adapt CodeCatalyst?
Please drop me a message on socials or an E-Mail!

Visits: 169

Amazon CodeCatalyst’s integration with the IAM identity center allows SSO with your own IdP

In this post you will learn how the new integration of the IAM Identity center with Amazon CodeCatalyst allows to use your own IdP as single source of truth of users for CodeCatalyst spaces. We will look at how you can connect a space to an IdP and on how this can be used to set up certain permissions based on your IdP roles.

Single Sign On (SSO) using IAM Identity Center empowers CodeCatalyst users to use their own IdP to provision user accounts in CodeCatalyst

With the new integration for Amazon CodeCatalyst into the AWS Identity Center organizations can now connect to their own existing IdP (AzureAD, Ping, Okta, …) going through the AWS Identity Center.

Benefits of the new integration

With this new integration users and organizations go to the Space settings in CodeCatalyst and choose to integrate with an existing instance of the IAM Identity Center.

This also connects potentialy to any other IdP that uses the IAM Itentity Center as a proxy. Administrators will only need to manage roles and permissions in on central place. The roles from the IdP can then be mapped to certain team roles in Amazon CodeCatalyst.

What I don’ t like about the new integration

There are two things that I didn’t like when looking at this integration:

Why do we need to go through the IAM Identity Center and cannot allow direct SAML or OIDC integration for a specific space? With the “need” to have an IAM Identity center organizations also need to directly connect to an existing AWS Account to manage connectivity – and this once again brings back a “tooling” account that initially I was hoping to replace with CodeCatalyst.

The second thing that I missed was the possibility to automatically map existing roles to groups in Amazon CodeCatalyst – today this is a – one time – manually effort that needs to be conducted. If you need to do that only once for your organization that isn’t a problem, but if you decide to have multiple spaces for your organization and need to perform this multiple times, then this can be very time consuming.

Summary

I believe that this change will have a very big impact on the adoption for CodeCatalyst as administrators and security teams in organizations will make it easier to allow the usage now that there is a central place to manage identities. And with the possibility to actually try out CodeCatalyst in an enterprise world, I hope that we will also see more adoption of CodeCatalyst and with that also a more streamlined and better communicated roadmap of CodeCatalyst.
If you would like to learn more on how to activate this, please refer to the documentation.

Visits: 73

Custom BluePrints in CodeCatalyst – templated projects that empower you to build better software

In this post you will learn how to build a Custom BluePrints in Amazon CodeCatalyst. You will get to know how Custom BluePrints allow you to generate a consistent setup of your projects and CI/CD workflows and how you can leverage this to empower your teams to be compliant with rules and deployment best practices. AWS has announced this today at his annual user conference AWS re:invent 2023 in Las Vegas.

Custom BluePrints – what they do, why they are there and how you can use them

Custom BluePrints can be seen as “project templates” that you can build and offer to all of your users of a space. As a user of CodeCatalyst you should already be aware of the general concept of a blueprint: It’s a templated project that you can start your software project on.

Custom BluePrints take this concept to a new level: Now you can build BluePrints for your projects!
And that’s not it: a project can be depended/generated by multiple BluePrints and a project generated from BluePrints can also become a BluePrint! yey!
You do not start from zero and you are now also abled to add a BluePrint to an existing project.

Think about what this means for you and your organization: You can set up certain default workflows, permissions, dependencies, environments, etc. and apply that to all of your new projects by default, without the need to “touch” anything.

And that’s not “it” – there is more: CodeCatalyst also allows you to “update” projects that have been build from a Custom BluePrint!

Think about the scenerio where you ahve over 20 microservices that all follow the same CI/CD pattern, including governance rules and dependencies. They all have the same dependencies and pre-requisites (e.g. AWS account permissions, environment setups, etc.). Now one of the dependencies has a critical, security relevant defect and you urgently need to upgrade all of these projects…

CodeCatalyst Custom BluePrints enables you to do exactly that with just a few clicks and pull request approvals!

I hope you are curious to look at this new functionality in details now, keep reading!

The technology behind blue prints – the power of Projen and Code Generation

Custom Blue Prints are powered by Projen under the hood – to learn more about Projen, please look here. Similar to a projen.rc.ts you will need to create a blueprint.ts file in the src folder of your BluePrint project. There you can then define the rules and automations which will be applied when the BluePrint is used as a starting point for a new project. Currently the Custom BluePrint SDK allows you to define Wizard configurations, Workflows, Environments, Source Repositories, Pre-Configured Dev Environments, … Using Projen as an underying technology the team is able to re-generate the code for your project and create a pull request on your behalf for the source repository of your project i there is changes. And this is great!

Building your first BluePrint

Building your first BluePrint is easy!

Create your BluePrint project

It starts with creating a new Custom BluePrint project in the CodeCatalyst UI using the BluePrint builder. This one can be accessed from the Space “Settings” page. After going through the wizard, this will create a new project for you in CodeCatalyst. In this project, you can now either checkout the source repository into your local IDE or you can use the DevEnvironments to directly get you started to work on the project.

Building and Editing Custom BluePrints can be a HIGH RISK

As you are reading this and as the functionality has only just been announced, it might be early – but it is never too early to warn about this:

If you edit a Custom BluePrint and potentially introduce security problems – maybe even unsecure dependencies – you might expose everyone that uses your BluePrint to challenges.

Better set up additional protections for all of your BluePrint projects!

Edit the sources of your BluePrint Project

Now that you your project is set up, you can start building your own components and parts of your project.

Be cautious to edit package.json in your BluePrint project – you can, but it might break some of the integrations. The typescript project is set up for you to be able to preview and publish your BluePrint.

The main “sources” of your BluePrint and the definitions will be in the src/blueprint.ts file. Initially, the project comes with a simple wizard set up with only a few parameters. It copies only the contents of the static-assets directory when being executed.

I’ve not been able yet to try out a lot of the functionalities and possibilities of the SDK, but still I was able to create a custom BluePrint that can be used to deploy a Flutter Web application with a Serverless Backend.

What I found out is that it will be a huge effort to set up the BluePrint in combination with a Wizard. This is not a 4 hour task, we’re rather talking about a week to get started. Further details on this – see the “Testing” section. This might also explain why this functionality is part of the new Enterprise tier for Amazon CodeCatalyst.

Please read the available options of package.json, but to get you started: use npm run blueprint:synth or yarn blueprint:synth to generate the BluePrint locally.

This will also execute any local unit tests that you have build for your BluePrint.

Testing your Blue Print locally

Using npm run blueprint:synth or yarn blueprint:synth will generate the BluePrint bundle locally and execute unit tests. It will generate a version of your project created by the BluePrint in the folder synth/synth.[options-name]/proposed-bundle/. This will use the default options you provided.

It is also possible to simulate the wizard adding the --cache parameter.

You could also build Unit Tests or SnapShots tests and integrate them into this lifecycle step of the build procedure.

Publishing your BluePrint to the Space Catalog

Use npm run blueprint:preview or yarn blueprint:preview to upload a “preview” version of your BluePrint to your CodeCatalyst space. This will make the Custom BluePrint available but not make it available in the catalog yet, so users cannot use it.

It also automatically increases the version number for your BluePrint.

Testing your Blue Print through the UI

After publishing your BluePrint preview from your local environment, you will now see this preview version within your Space Settings. From there, you are able to create a project out of the Preview version of your BluePrint.

Here you will need to test all options of the wizard and in addition to that verify that the customizations that you added have been correctly generated after you went through the wizard.

I have made the experience that a whole testing cycle can take 3-5 minutes – so I really recommend you to look at local unit tests.

After you have finished your tests, you can add a specific version of your BluePrint to the BluePrint catalog. This is currenty only be possible through the UI in the Space Settings.

Creating (one or many) projects from your BluePrint

Now you are ready to start creating projects from your BluePrint!

Go to the normal “Create Project” screen – after using “Create from Blueprint” you will now have the option sect “Space Blueprints” which is where you will find your newly created BluePrint.

This should now generate your project from your BluePrint and the structure should be as you wanted it to be. 🙂

You can also add additional BluePrints to an existing project – this allows you to incorporate multiple BluePrints in a single project, a combination of multiple “default” project settings.

Please be aware that this will remove manually added or created resources that are part of the BluePrint that you are adding – it will overwrite manual changes that “interfere” with the BluePrints code.

Upgrading your BluePrint

Upgrading your works similar to creating it – upload a new “Preview” version, add the new version to the catalog.

Upgrading your projects to a new BluePrint version

You can currently update the used Blue Print vesion for a project by going to the project settings, to the Blue Prints section there and then by loading up one BluePrint. Choosing to upgrade to a different or newer version will generate the proposed changes and make them visible to you before you apply them.

What’s next for Custom BluePrints?

Oh wow, this is one of the new features that I am most excited about for CodeCatalyst! There is so much potential! I have to explore further the possibility to have multiple BluePrints attached to a project, the possibility to remove a BluePrint from a project, and all of the other things that I didn’t touch yet.

Here’s stuff that I would love to get for Custom BluePrints:

  • auto-apply updates to ALL projects that are from the same BluePrint automatically
  • share Custom BluePrints with other Spaces
  • a Custom BluePrints market place where developers (like me) can put “their” BluePrints in and organizations can “buy” a BluePrint from

I also believe that the team needs to work a bit more on the developer experience when building Custom BluePrints, e.g. by pre-suggesting automated integrated tests (integration or snapshot tests). Publishing a new version to the catalog should also visualize the changes that are going to be added to other projects before you actually publish the new version.

I am a bit disappointed that the “free tier” does not give you the possibility to try out this functionality by being able to add one or two Custom Blue Prints to your space, but maybe this is something that will be possible as soon as the Blueprint SDK is stable.

What do you think of Custom BluePrints? What are your main wishes? Please reach out to me and let me know!

Visits: 372

A real project with CodeCatalyst – Our hackathon gave us a good insight into what works and what doesn’t

As Matt already wrote in his project introduction post we have been using Amazon CodeCatalyst to handle all of our project activities. This gives a very good indication into where CodeCatalyst is already “ready for prime time” and where further adjustments need to be made in order to develop the service into a fully fledge, all-or-nothing, integrated DevOps service.
The one pushing for using CodeCatalyst was myself, as I wanted to try out the new Amazon service that recently went “GA” after being announced at re:invent 2023 in a bigger team and in a real project. It was great fun – and we learned a lot, too!
Let’s look at some details.

How we planned our tasks using “issues”

Our journey with CodeCatalyst started with a – very agile and intense – planning session using the “issues” component of CodeCatalyst. Given the nature of an hackathon, we created roughly the first ten issues in the system. Three of us were “new users” for CodeCatalyst, but they were able to quickly interact with the issues component – by default, this is a simple Kanban board and has a well structured user interface. One thing we directly missed was the ability to copy/paste images into comments within the issues…and to attach files to the issues themselves. Linking issues to one another is also only hardly possible right now.
We used a simple workflow and did a regular planning (twice per week) and created/updated/closed more than 50 issues (of partly very small scope) throught the course of less than four weeks. Overall, the available functionalities were good enough for our project and supported our project.

Collaborating source code – working with Git repositories and pull requests

The four of us are (Community) Builders from the bottom and all we need to be happy is a Git repository that we can push code to 🙂
Of course this is not currect, but CodeCatalyst allows you to host you own, private Git repositories. You can have multiple repositories per project. In our case, we had decided to go with a mono-repo approach and so we strickly worked in a single repository. Working with the source code repository was “just fine” and working as expected. We also decided to merge as early and as often as possible to our “main” branch. The team however struggled in working with setting up Pull Requests and working in the UI when reviewing pull requests.
The most problematic things: UI response time, working with comments on pull requests (e.g. threaded comments) and creating pull requests. The whole UI does not yet feel as “structured” as in other competitors (e.g. Github) – examples: PR title/description proposals when creating a PR, link in terminal/commandline that allows your to create a PR.
Another thing we missed is “git blame” online – not to blame each other but to understand what changed and broke our “hacked” “hackathon” source code 🙂
Cool: the markdown rendering for files named *.md. Missing here: including local images is not possible.

CodeCatalyst usage increased a lot in this project!

Working with Workflows – Github actions for the rescue!

We started off building our workflows with natively available actions in Amazon CodeCatalyst – using the “cdk deploy” action to deploy out infrastructure and the “s3 deploy” action to deploy our front end – but quickly shifted away from that and switched over to using Github Actions within the workflow to allow building our Flutter application as a CDK asset. As our Continuous Integration pieces matured within our npm implementation, there was less reason to go back to the native actions as the CI part was handled through npm.
Our current workflow and CI/CD pipeline:



The main preference here is to allow a continuous deployment to our production environment. This goes through an intial “sandbox” deployment, promotes to a “test” environment and then deploys to “production”. The pipeline is prepared to execute integration tests, but they are not yet implemented – which is the nature of a _hack_athon 🙂
This would definately one of the next things to change – adding real integration tests and also adding some security verifications.

Our workflow is, as I mentioned, pretty basic – here an excerpt of it until the deployment to our sandbox environment:

BuildAndTestCDKApp:
    Identifier: aws/github-actions-runner@v1
    Compute:
      Type: EC2
    Inputs:
      Sources:
        - WorkflowSource
      Variables:
        - Name: AWS_REGION
          Value: us-east-1
        - Name: NODE_ENV
          Value: sandbox
    Outputs:
      AutoDiscoverReports:
        Enabled: true
        ReportNamePrefix: rpt
        IncludePaths:
          - ./coverage/lcov.info
    Configuration:
      Steps:
        - name: Flutter Build Web
          uses: subosito/flutter-action@v2.8.0
          with:
            channel: stable
        - name: Java Install
          uses: actions/setup-java@v2
          with:
            distribution: zulu
            java-version: "11"
        - name: Setup Android SDK
          uses: android-actions/setup-android@v2
        - uses: actions/setup-node@v3
          with:
            node-version: 16
        - run: npm ci
        - run: npm run
        - run: npm run synth
    Environment:
      Connections:
        - Role: CodeCatalystWorkflowDevelopmentRole-AWSCommunityBuilders
          Name: community-builders-sandbox
      Name: speakers_sandbox
  DeploySandboxCDKApp:
    DependsOn:
      - BuildAndTestCDKApp
    Actions:
      DeployBase:
        Identifier: aws/github-actions-runner@v1
        Compute:
          Type: EC2
        Inputs:
          Sources:
            - WorkflowSource
          Variables:
            - Name: AWS_REGION
              Value: us-east-1
            - Name: NODE_ENV
              Value: sandbox
        Configuration:
          Steps:
            - name: Flutter Build Web
              uses: subosito/flutter-action@v2.8.0
              with:
                channel: stable
            - name: Java Install
              uses: actions/setup-java@v2
              with:
                distribution: zulu
                java-version: "11"
            - name: Setup Android SDK
              uses: android-actions/setup-android@v2
            - uses: actions/setup-node@v3
              with:
                node-version: 16
            - run: npm ci
            - run: npm run deploy
        Environment:
          Connections:
            - Role: CodeCatalystWorkflowDevelopmentRole-AWSCommunityBuilders
              Name: community-builders-sandbox
          Name: speakers_sandbox
  IntegrationTestSandbox:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: ls -al
    Compute:
      Type: Lambda
    DependsOn:
      - DeploySandboxCDKApp

As you can see, a lot of the CD (and also the “deployment”) runs within the corresponding task definitions in package.json.

No “Open Source” or “Readonly” view

We would really love to open-source our whole projet, to give you the possibility to get involved and learn from what we build, but unfortunately Amazon CodeCatalyst does currrently not offer an Open Source or a “Readonly” view and forces you to create an account for CodeCatalyst – these are things that make it difficult for us as we cannot link to our sources.
If you are an AWS Community Builder, please reach out to either of us in Slack and we can give you access to the project.

What did we really miss in the workflow component

  • manual approvals before promotion to our “production” environment
  • Notifications from workflows – could have been done through this implementation but we did not take time to implement that
  • Integration of actions deployed through “npm – cdk deploy” with making the changes in infrastructure visible
  • importing existing pipelines or render CDK pipelines to CodeCatalyst workflows The “killer feature” of the workflows, at least as far as we see it, is the possibility to use Github actions as part of the workflow. With only minor adjustmends an existing Github workflow can be transformed to a CodeCatalyst workflow. If either of you know about an existing tool that converts existing Github workflows to CodeCatalyst workflows in an automated way – please let me know.

The hidden gem: Development Environments

One of the “hidden gems” of CodeCatalyst are the Dev Environments – also known as Development Enviroments. Essentially, its a “service in the service” similar to Gitpod where you can host your IDE or the environment that you develop on. In our project we all didn’t use it as the architecture was small and we only had this single project. Also, we were developing mainly on our main machines without switching between machines too often. That’s why we all did not consider really using the DevEnvironments for this project. Last but not least, Flutter is not natively supported on the Dev Environments which would have forced us to prepare a devfile that includes the Flutter dependencies – and thats something that would have taken too much time out of the project.
Still, the De Environments within CodeCatalyst are hidden gems that I believe should get more traction than they are getting until now.

A summary of what we think works and what doesn’t – and our biggest wishes

For a hackathon like this (3-6 weeks working time) CodeCatalyst seems to be a very good choice for a project that starts with a “new” or “fresh” codebase. The tool has most of the minimal requirements implemented to allow simple project management and good integration with AWS.
It also allowed us to quickly get started and to collaborate on our sources.

We really missed workflow notifications as well as manual approvals in the workflows.
As the tool matures I personally hope that more and more AWS internal teams will be using CodeCatalyst to develop, plan and deploy out their internal AWS services and with that the “flow” in the User Interface will definately be improved, too – as today there are some hickups in a real developers workflow.

And, last but not least – we would love to share this project with you, but given CodeCatalyst does not allow to “Open Source” a project, that’s not possible 🙂

Maybe something for the next hackathon?

If you would like to get involved in this project and help us to shape and build our AWS Speakers directory to make it easier for AWS User Group Leaders to find speakers – please reach out to either of us four about collaborating!

Who to reach out to?
Danielle, Matt, Julian or myself.

Visits: 922

Amazon Codecatalyst reaches “GA” status and becomes available for general use

The new service announced by Amazon in Las Vegas at re:Invent 2022 which is an integrated DevOps service to empower development teams to develop and deliver software faster finally reaches the “general availability” status. As I have previously outlined, this achievement is very important for Amazon and the CodeCatalyst team. Congratulations to the team for reaching this goal, which I can imagine is not an easy step for this product. The tool touches a lot of very sensitive parts of a software project and I can imagine the security standards being really high. 

A hugh achievement – thank you everyone in the team for investing into CodeCatalyst and for listening as closely to the customer feedback as you are!

What changes did get implemented for GA?

As part of the GA release we see a lot of minor improvements in the User Interface and color changes. In the last weeks, we have seen a few “bigger” changes – like the possibility to use Dev Environments for Github based projects. We also got “graviton based” execution environments for CI/CD workflows which, according to AWS, should reduce our costs.

It is still hard to track down all of the changes in CodeCatalyst, as there is – to my knowledge – no public or semi-public roadmap. This is one of the things that I’d love to see, as for an integrated service that is at the core of the Developer Experience for teams, any minor change can either improve or destroy the “usage experience”. If you as a team invest into adoption a new tool like CodeCatalyst, they will need to know how changes in workflow, features or user interface can influence their day-to-day activities. Let’s see, maybe the team can share “something” like a “changelog” with us (or even an RFC process like Amplify or AppSync)?

Reached “GA” – so who can start using it now?

As of today CodeCatalyst is only availble in US regions and this means that it can be adopted mainly by US enterprise customers. CodeCatalyst already gives you the possibility to set up different Spaces for your account and within a space you can manage multiple projects. So in theory, CodeCatalyst is “ready to be used” by everyone. 

Practically speaking, it is easier to adapt the service for new projects than for existing projects , as there is no real “import” functionality. Yes, you can integrate existing Github projects, but that only integrates the source code. Unfortunately that does not make all of the “cool” things available right from the start of integrating the source: existing workflows (CI/CD pipelines) are lost and need to be re-build, issues/tickets are not imported into CodeCatalyst (tho they can be made available through the JIRA integration). 

I have been regularly using CodeCatalyst (both for imported and “new” projects) – and I really think that the tool already works very well. 

The “killer feature” that I see for new projects are the “blue prints” which essentially get you started within minutes, e.g. to deploy a SPA application, or to have a “true” CI/CD pipeline for a full stack application following the DPRA. 

Right now I would recommend using CodeCatalyst for any new project that you start to start building out your workflows and best practices.

So what do I still need to recommend CodeCatalyst for existing projects?

There are a few things that I have already been writing about:

  • “Import” of existing CI/CD workflows (e.g. Github actions, CDK Pipelines or CodePipelines)
  • Fully import projects
    • existing issues from Github or JIRA
    • Git-based projects including the history
  • Tighter security settings and permissions
    • Fine granular roles to allow or forbid access to specific parts of a project
    • Options to allow or forbid execution of workflows (or to deployments)
  • Additional workflow options
    • Manual approvals are very high on my wish list
    • Integration of other AWS services natively

A question for the readers: What do YOU think that you need to adopt CodeCatalyst?

A big question for the CodeCatalyst team – HOW MANY AWS TEAMS ARE USING CODECATALYST FOR PRODUCTION DEPLOYMENTS TODAY?

Where do I see the potential for CodeCatalyst?

CodeCatalyst is a big bet by AWS. There is a big potential that can really improve the life of development teams and these are the main things that I believe that can out-grow other existing solutions:

  • Integration of AWS Services / deployments metrics
    • the true integration with AWS APIs
    • Integration into “post-deployment” verifications (e.g. auto roll-back after failed CloudWatch metrics)
  • “At-hand” developer support to improve efficiency
    • with CodeWhisperer (who recently reached GA) AWS already aims to support developers during the development phase, but with CodeCatalyst AWS can take this to the next level:
    • AI support during Pull Request Reviews (or automated approvals for PRs – e.g. by including CodeGuru, etc., automated merges, etc.)
    • AI support during workflow executions (when to approve, when to deloy, when to promote, etc.)
    • With improvement proposals for workflows if the “AI model” recognizes patterns (in issue workflows or CI/CD workflows)
  • With automated improvements for existing projects based on blue prints
    • Best practices change – and so blue prints change – and if the CodeCatalyst team can automatically apply them for existing projects, customers will benefit from it

And last but not least:

I trully believe that every software project should start with a CI/CD pipeline – and with the Blue Prints including the CI/CD workflow that follows DPRA and other AWS best practices, we can trully make this possible: Empower developers to deliver their software projects in minutes right after starting their project.

Do you see the potential in CodeCatalyst? If you do not see any potential in the tool – why not?

Visits: 1626

What can we expect for the General Availability (GA) of Amazon CodeCatalyst?

At re:Invent 2022, as usualy, different new AWS Services or functionalities have been announced in Preview. Now, at the beginning of April 2023, a few of them have already reached the “General Availability” (GA) status – Application Composer (in early march), Latice (in late march). My favourite new service, Amazon CodeCatalyst, has not yet reached this goal – but I have a feeling that now is the right time to think about what and when we can expect this status.

You wonder what CodeCatalyst is? Watch this video on my YouTube channel or read my two initial posts about it.

Why is reaching the “GA” milestone so important?

Before starting with my assumptions on what we can expect for GA, lets clarify why reaching this milestone is so important. Being “in Preview” can mean a lot of different things. In a lot of organizations this usually translate to “limited availability”, a service not being available in all regions or not being reliable or scalable. For other organizations, it means that specific aspects of the product can be immature or not reliable. It can also mean that bigger API change are yet to be implemented or missing security guardrails. 

In general, this can be seen as a “beta” offering which is not appropiate to use for productive workloads.

Because of these reasons and maybe others, a lot of organizations (especially US based) do not allow using or adapting services that are in “Preview”.

For all of my experiences, tests, videos and projects I was so far able to only be on the free tier. And I assume that this will also be the truth for most of my readers: You can get a long way using the Free Tier that Amazon CodeCatalyst offers today.

So thats another big reason for AWS to push this service out of “Preview”: It gives organizations that are forbidden to use the service in “Preview” the possibility to start using and adopting the service – and with that Amazon can start earning money with the service which unil now might be difficult.

And as we know, AWS tries to “work backwards” from client requirements and the early usage of CodeCatalyst will drive further investments into the service.

What to expect for GA of CodeCatalyst?

Simple: Nothing big – most probably only regional rollout.

I do personally not expect any major new features for the service as the team has been constantely releasing new features and functionalities to the service on a regular cadance. There was simply not more time to work on bigger features while preparing the “General Availability” (GA).

What the CodeCatalyst already has delivered until today…

Let’s look at what has been added to CodeCatalyst since its official release in december 2022:

  • Additional Reporting auto-discovery
  • Change Tracking – the possibility to see which changes have been deployed to a certain environment
  • Additional Workflow native actions and improvements, E.g.
    • a problem with the CDK action to be able to define the “workpath” of a CDK app
    • Additional native actions
  • Linked issues to Pull Requests – you are now able to link issues to a pull request
  • UX improvements
    • Log files wider accessible in UI – at the beginning you where not able to make the log view larger, now this is possible
    • Page title adjustments
  • New Blue Prints (like the “Textract” one)
  • Development environments for Github based projects

This is not a complete list, but the things that I personally noticed and that I liked to see.

So…when is “the date”?

Hard to guess, but I would expect “soon”. Ideally right before a month starts, which will make the billing cycle easier 🙂 

So I would guess “end of april” which would bring the service right in time for the Berlin Summit (3rd of May).

Next steps for CodeCatalyst

In my last posts I have already been communicating my thoughts and features that I would love to see. But what will AWS implement?

Given reaching the “GA” status opens the way to “enterprise clients” I would expect that one of the first features will be Single-Sign-On functionalities, maybe with an integration to Okta, Ping, Azure Active Directory or other already existing IdPs.

In addition to that I believe that the User Interface needs to get some tweaks to streamline the navigation and workflow – that’s something that I personally experience every day: not knowing when and where to click to get on the rigth place. Also I think that additional service integrations will be added – e.g. StepFunctions or SNS, maybe SQS – see also my post about sending notifications from workflows.

And then there is one last thing which has been getting only limited attention so far: APIs and CLI integrations that can be used – so I would expect a major update there.

I’m really looking forward to see CodeCatalyst reaching GA – I’ve had various conversations with the team in the last months and I know that they have a true vision to make CodeCatalyst successfull as a trully AWS integrated and fully functional DevOps tool.

Are there features you are missing? Please let me know and I will forward them to the team.

Visits: 730

Sending notifications from CodeCatalyst Workflows in March 2023

As Amazon CodeCatalyst is still in Preview, it has only limited integration possibilities with other AWS services or external tools.
Sending notifications from a Workflow execution is something that I believe is critical for a CI/CD system – and as I focus on CI/CD at the moment I’ll focus on the notifications on Workflows in this article.

What kind of notifications do I need or expect?

As a user of a CI/CD and Workflow tool there are different levels of notifications that I would like to receive:

  1. Start / End and Status of Workflow execution
  2. State / Stage transitions (for longer running workflows)
  3. Approvals (if required)

In addition to that, based on the context of the notification I would like to get context-specific information:

a) For the “Start” event I would like to know who or which trigger started the workflow, which branch and version it is running on, which project and workflow has been triggered. If possible getting the expected execution time / finish time would be good
b) For the “End” event I would like to know how long the execution took and if it was successful or not. I would also like to know if artifacts have been created or if deployments have been done. If the “End” is because of a failure, I would love to know the failure reason (e.g. tests failed, deployment failed, …)
c) For the state transitions I’d love to know the “time since started” and “expected completion time”. I would also like to, obviously, know the state that has been completed and the one that will now be started.
d) For approvals I’d love to be able to get the information about the approval ask and all required information (commit Id, branch) to do the approval

What does CodeCatalyst Support today?

Right now CodeCatalyst allows to set up notifications to Slack.
Please see details on how to set this up here.
This notifications are also minimal right now:

In Slack this looks like this:

How can I enhance the notification possibilities?

Luckily one of the “core actions” is the possibility to trigger a Lambda function and this is what we are going to use here to be able to trigger advanced notifications using Amazon SNS.
In our example we are going to use this to send an eMail to a specific address, but you can also use any other destinations supported by SNS like SMS or AWS ChatBot.

Setting up pre-requisites

Unfortunately we will need to set up an SNS topic and a Lambda function in a dedicated AWS account in order to use these advanced notifications.
This means that we are “breaking” the concept of CodeCatalyst not requiring access to the AWS Console, but this is the only way that I found so far to be able to send additional notifications.

Ideally we would be setting up the SNS topic and the lambda function using CDK, but that increases the complexity of the workflow and of the setup and because of that I’m not including that in this blog post.

Setting up the SNS topic

Please create a SNS topic following the AWS documentation through the console.
We assume the topic to be in “eu-central-1” and the name to be “codecatalyst-workflow-topic“.

After the topic has been set up, you will need to subscribe your eMail address to it.

Setting up the lambda function

You can follow this blog post to manually set up the lambda function through the AWS console, please ensure to give the Lambda functions permissions to use the SNS topic.
The required code using Python will look like this:

import boto3

sns = boto3.client('sns')

def lambda_handler(event, context):
    try:
        message = event['message']
        topic_arn = 'arn:aws:sns:eu-central-1:<accountID>:codecatalyst-workflow-topic'

        response = sns.publish(
            TopicArn=topic_arn,
            Message=message
        )
        print('Message sent to SNS topic:', response['MessageId'])
    except Exception as e:
        print('Error sending message: ', e)

Obviously the same can be achieved using Typscript, Go or any other supported function.
Please adjust the topic_arn to match the topic that you just created.
After creation this Lambda function will now have an ARN which should look similar to this:
arn:aws:lambda:eu-central-1:<accountId>:function:send-sns-notification-python

We will need this ARN when setting up the notification in our Workflow.

Integration into the workflow

Integrating this Lambda function into a workflow is easy:

  NotifyMe:
    Identifier: aws/lambda-invoke@v1
    Environment:
      Connections:
        - Role: CodeCatalystPreviewDevelopmentAdministrator-wzkn0l
          Name: "<connection>"
      Name: development
    Inputs:
      Sources:
        - WorkflowSource
    Compute:
      Type: Lambda
    Configuration:
      RequestPayload: '{"message":"branchName: ${WorkflowSource.BranchName}\nCommitID: ${WorkflowSource.CommitId}\nWorkflow-Name: NOT-AVAILABLE\nSTATUS: EXECUTED"}'
      ContinueOnError: false
      AWSRegion: eu-central-1
      LogType: Tail
      Function: arn:aws:lambda:eu-central-1:<accoutId>:function:send-sns-notification-python

As you can see, we are integrating an “aws/lambda-invoke@v1” action which then points to the lambda function that we just created.

In the “RequestPayload” we are passing a few information to the Lambda function which will then be passed to the SNS topic as part of the message.
This is how the message will look when received as eMail:

Missing information and next steps for enhanced notifications

As you can see we are able to send notifications from CodeCatalyst to multiple targets, including eMail with this option.

What we are missing is – and I am not sure if thats possible or not – is all of the “metadata” of the workflow execution like:

  • Workflow-Name
  • State-Name
  • Project Name and additional information

In the documentation I was not able to find out the available environment variables about these information…. If you do have any ideas on how to access this metadata, please let me know!

Visits: 623