Tag: aws

The modern CI/CD toolbox: Strategies for consistency and reliability

Introduction

Welcome to the blogpost supporting the AWS re:Invent 2024 session “DEV335 – The modern CI/CD toolbox: Strategies for consistency and reliability”.

We aim to not only summerize but also enhance your session experience with this blog post.

Please do not hesitate to reach out and ask any questions that you might have in the comments or reach out to us directly on socials.

If you’re an old-school person, reach out to us by eMail.

Session walk-through and contents

CI/CD foundations

Continuous integration (CI) involves developers merging code changes frequently, ensuring that the cumulative code is tested regularly—preferably multiple times a day. Continuous delivery (CD), on the other hand, requires manual approval before deploying to production, while continuous deployment is entirely automated.

Unified CI/CD pipelines

Thorsten emphasized the importance of having a single, unified pipeline for all kinds of changes—whether they are application code updates, infrastructure modifications, or configuration changes. This helps maintain consistency, reduces risks, and simplifies compliance.

Code signing and attestation

According to Gunnar, ensuring the integrity of the code through signing and artifact attestation is paramount. This practice verifies that the code hasn’t been altered improperly, tracing each change back to a trusted source, which significantly reduces the risk of tampering and supply chain attacks.

GitOps: a new look at operations

Johannes took an in-depth look into how GitOps integrates Git with operations, streamlining deployment decision-making. GitOps supports a fast, automated transition into production environments without manual intervention, making it powerful for Kubernetes and other cloud-native projects. The main takeway is that with implementing GitOps, the decision to deploy a change to production is taken by the team members close to the context of the change instead of being taken by a “Change Advisory Board” or managers that are far away from the actual change.

Deployment strategies for minimizing risks

Several deployment strategies, including rolling deployments, blue-green deployments, and canary deployments, were outlined by Gunnar. Each strategy offers a different balance of speed and risk, with options to revert back to previous versions quickly if issues arise. You will need to choose the strategy the fits your business needs and your applications requirements.

Drift – Avoid it at any option

In this section, Johannes highlighted the challenges that come around with “Drift” in deployments – which is defined as any kind of manual changes that are done in your cloud deployment without going through Infrastructure as Code (IaC) and CI/CD. We gave the guidance to ensure that noone should get access to the target account to perform manual changes, instead, you should implement a “break-glass” pipeline that is focused on speed to recover form application downtime by rolling forward through CI/CD.

Ensuring consistency across pipelines

Torsten introduced an innovative approach to maintaining pipeline consistency using constructs. By centralizing the standard pipeline templates and allowing teams to extend them, organizations can adapt to specific needs without sacrificing consistency. This method also assists in managing migration between various CI/CD platforms effectively.

The role of security and compliance

Security and compliance are non-negotiable, integral parts of any CI/CD process. Integrating these practices from the beginning ensures that both security and compliance standards are maintained throughout the development lifecycle.

Feature flags and progressive delivery

Gunnar highlighted the importance of feature flags and progressive delivery in decoupling deployment from feature activation. With feature flags, changes can be made dynamically without redeployment, enhancing agility and reducing risk. This approach, used by companies like Netflix, enables controlled risk management and early detection of issues.

Avoiding vendor lock-in with projen-pipelines

Thorsten presented a possibility for CI/CD practicioners to adapt an open source project called projen-pipelines that empowers developers to switch between different CI/CD vendoers by allowing to define the pipelines in Typescript and implementing a renderer process that is able to generate pipeline code for Gitlab, Github, CodeCatalyst and bash.

Conclusions

The insights from this session highlighted the ever-evolving nature of CI/CD practices, where automation, innovation, and stringent security measures play crucial roles. As we continue to refine these practices, it’s clear that the right blend of technology and methodology can significantly impact the efficiency and reliability of software delivery processes.

Next steps

To dive deeper into these strategies, check out the resources and links provided below. Engage with the wider community to exchange ideas and best practices, and continue evolving your CI/CD processes to meet future challenges.

Thank you for attending this session and for taking the time to read the additional information provided here.

Please do not hesitate to reach out and ask any questions that you might have in the comments or reach out to us directly on socials.

If you’re an old-school person, reach out to us by eMail.

The links mentioned in the session

Views: 523

Dear AWS, how do I build & develop purely on AWS right now?

The announcements from AWS around deprecating certain services have raised a bunch of questions and concerns in the AWS community. 

As Jeff Barr wrote, these are the services:

S3 Select, CloudSearch, Cloud9, SimpleDB, Forecast, Data Pipeline, and CodeCommit.

This post will focus on Cloud9 and CodeCommit … and how I think this announcement impacts the “end to end” developer story for developers on AWS. We’ll also look at how the announcements impacts my “Go-To” services Amazon CodeCatalyst.

It is written from the perspective of a builder that mainly uses AWS tools for smaller side projects and can be seen as a “startup” that needs to quickly be up & running without much hassle.

Introduction

These annoucements, and the way that these deprecations where announced

Blog for CodeCommit

Blog for Cloud9

are in my humble opinion one of the worst possible ways. I know the teams at AWS have seen the feedback and I hope that there will be a clearer communication strategy going forward.

For me the combination of those posts with the assumption of CodeCatalyst being built on top of these services gives a very strange feeling on how much AWS is currently invested into Developers on AWS.

Let’s look at why I see a lot of impact of these announcements for builders and think about alternatives if you are using CodeCommit or Cloud9 for certain aspects today.

Tools required for SDLC

A few weeks ago I even dedicated a complete Shorts-Playlist on all of the Code* tools, looking at their usage and the approach to cover a full Software Development Lifecycle (SDLC) building on AWS.

In this series I drafted the diagram:

AWS Tools part of your SDLC until recent announcements

This being and end-to-end flow, AWS had at least two options to implement this process using their tools:

Either CodeCatalyst or a combination of different AWS Services

When CodeCatalyst was announced, I wrote about how CodeCatalyst can be used to cover all parts of your Secure Development Lifecycle (SDLC) process on AWS. Ever since then, there was an alternative on AWS using a combination of different building blocks: CodeCommit, CodeBuild, CodeDeploy, CodePipeline and others.

CodeCommit was a good, reliable managed Git server. For the purposes it solved, there weren’t many features to add. It was a managed service you didn’t need to think about and just “serve its purpose”.

Cloud9 was a hosted IDE, a development environment that users where able to access through their browser. This enabled builders to have a real IDE, even on old or underpowered computers, anywhere — even when being on vacation.

Developers on AWS are still able to use CodeCatalyst to cover for all parts of your product lifecycle or they had the alternative to use the different “building blocks” to compose their SDLC process. Both options gave value and helped AWS customers to solve certain aspects and problems.

Now, officially, only one option is left — CodeCatalyst.
CodeCatalyst is an integrated DevTools service that unites all of the building blocks under an opinionated, structured user interface. It was announced at re:Invent 2022 and went GA in early 2023. With the custom blueprints feature, it also enables builders to create project templates and share them with their team mates or dependend teams. Very powerful possibilities for teams to collaborate better and also share their best practices with other teams.

Those that didn’t need a “reliable managed Git server” where most probably using existing alternatives — that might solve the “job” better than CodeCommit — like Github, Gitlab or Atlassian. These users and AWS customers are not affected by the change.

What has changed with the July 2024 announcements — builders perspective

Now, the system landscape has changed.

Developers can not use Cloud9 anymore to develop software, they need to fallback to alternatives like Github Codespaces, Coder or Gitpod.

Developers cannot store their source code in CodeCommit anymore, they need to fallback to alternatives like Github, Gitlab or Bitbucket.

And given CodeCatalyst might be using CodeCommit under the hood and is using Cloud9 for the DevEnvironments – Can I really build something on top of CodeCatalyst going forward?

So this announcement of deprecation — without a “real” AWS native alternative — puts everyone building and developing software on AWS in the situation of needed to look for alternative setups.

Especially it forces you — if you are a small organization (or a startup) to engage with more than just one vendor as part of your SDLC lifecycle and process. I see this as a critical point to talk about aswell.

And, if you are building software or platforms on AWS where especially CodeCommit is part of application or the deployed architecture itself — You are now left without any option. If you want to integrate a Git server in your application on AWS, you will now need to self-host the git-server instead of using a managed service.

If you “just” needed a Git repository, quickly, fast and reliable — CodeCommit was the way to go. Now, you need to use a 3rd party alternative.

Now: What options on AWS do we have as builders?

What changed with the July 2024 announcements — business perspective

Looking at the announced changes from a different perspective, we need to acknowledge that AWS is a 90+ billion (90.000.000.000) dollar company. It is clear, being a business that aims to “make money”, that AWS needs to focus on services and solutions that are widely used, adapted and earn a good margin.

The reason might be that Cloud9 and CodeCommit where just not profitable enough to drive the expected growth of the business. Especially, as there are other services that do the same job better than Cloud9 and CodeCommit. So it might have been “just” a business decision to stop investing into these services and focus instead on Amazon Q that promises to help developers and builders on AWS.

This raises the question on which other services might soon or in the future be hit by exactly the same challenge. And – how is “success” measured by AWS on services? Is it “just” revenue or are there other points that are being considered?

But still — How this feels for me and questions I have (emotionally)

It feels like AWS has given up the game of engaging with their “Builders” and is now focused on the “Buyers” that “host” their applications on AWS.

If you think about how AWS started and if you look at how much effort AWS has spent this year on making us think that “Amazon Q Developer” is going to make our lives as developers easier…

How can I as an advocate for AWS as a platform be confident that I am valued as “Builder” on AWS? Will other services also disappear if they do not get enough traction?

And how much can I trust in Werner’s “Now, go build“?

How much “trust” can I put in the other Code* (CodeBuild, CodePipeline, …) tools on AWS?
With CodePipeline and CodeBuild getting a lot of notable updates right now (macOS, Github action runners, Stages rollback, …) the outsiders view is that at least these services are there to exist… but how much trust has the AWS team lost with Builders around the globe?

I’m eager to see how the different workshops, best practice documents, open source projects that use either CodeCommit oder Cloud9 (especially also AWS owned) will be adjusted and updated in the next weeks and months.

How much is also CodeCatalyst going to be the central place for Developers on AWS? How much updates will we see there?

How does this affect you – I would love to know!

I am really interested to hear how these announcements have affected your perspective on AWS and your view on the different AWS services.

Please share your thoughts either as a comment to this post or reach out to me personally!

What YOU can do next

You could now follow the advice from AWS and “migrate” away from CodeCommit or Cloud9 — but is this really what you want to do?
If you have a need to have a “Git server” or “Git repository” close to your applications on AWS, how do you do that?
You might need to host your own Git server on AWS….or you need to give up on that premise and fallback to alternative Git providers like Github, Gitlab, …

If you insist on having your own hosted Git within your AWS environment, there a few possible solutions…

…and potentially others that I am not aware of….

In order to host a “simple” Git setup I’ve recently made this repository public that deploys Gitness as a Git Repository on ECS. It will cost you roughly 50 USD/month. See also a relevant blog post.
Inspired by this, Jakub Wolynko did the same thing for Onedev – please see https://github.com/3sky/onedev-on-ecs if you would like to try that out.

As an alternative for Cloud9, you can use vscode.dev, which runs VS Code in the browser or other alternatives that are more integrated and personalized like gitpod.io or Github Codespaces.

But is this REALLY what you want to do if you are working on AWS only?

What I hope to get from the AWS team

As re:Invent is approaching fast and that usually sets the direction for a lot of AWS services, I really hope to get reliable information and roadmap clarifications around the AWS developer tools.

I’d like to understand if I can rely on CodeCatalyst, CodePipeline, CodeBuild, CodeArtifact, CodeDeploy, … and other AWS services that help developers to build software on AWS.

Does anyone know if this page ever mentioned CodeCatalyst? Please let me know!

In addition to that, I would love to get a better and more detailed overview on what the level of support will be that customers of the “deprecated” services will get: Security Updates? Priority Support?
Creating one page that summerizes that for all “deprecated” services would be amazing!

And – last but not least – make sure that Amazong Q knows which services you are deprecating!

Screenshot taken on 6th of September, 4pm CEST

If you’ve read this post until here, I would love to get your view and your feedback on this topic!

Thanks for the feedback I got before publishing this article and while I know you don’t agree with everything I wrote, it’s great to get your feedback, Monika, Raphael, Ran, Markus and others 🙂

Please let me know either in the comments or directly on my social channels — LinkedIn, X being the ones I still use mostly 😉 

Views: 1040

A self-hosted CodeCommit alternative

A few weeks ago AWS CodeCommit became a deprecated service on AWS. This means, customers cannot create new repositories anymore – refer to this announcement for all details: Blog for CodeCommit

There are obviously a lot of alternatives to CodeCommit (Github, Gitlab, …) but if you need a “self-hosted” Git repository in your own AWS account this can become a little bit harder to provision.

If you’re looking for a “self-hosted” Git server, a bunch of tools come up:

  • Gitea
  • Gitness
  • Gitlab
  • …and obviously also others or just a “git” server running on EC2

As I wanted to be able to deploy something that works “out of the box” I looked at how to provision one of these alternatives on my own AWS account.

Gitness

Gitness is an open source development platform packed with the power of code hosting and automated DevOps pipelines. It’s biggest mantainer is Harness. It includes “way more” than just Git – eg Gitspaces, Pipelines, etc.

Deployment of Gitness

If you want to deploy Gitness, the docs point you at running it locally using Docker, or by deploying it on EC2 or k8s.

My aim was to “only” make the “Git” component available and because of that I’ve chosen Amazon ECS with EFS as storage to provision Gitness.

Open Source Code for the deployment of Gitness

I’ve set up a project on Github where all of the code examples are available for you to look at:

https://github.com/Lock128/setup-gitness

In the rest of the article, I’ll try to walk you through the most important parts of the project.

Please note that this code is not production-ready, it is more a PoC to showcase the direction that you could take.

Source Overview

We’re using AWS CDK and Typescript for Infrastructure As Code (IaC).

We have a “bin” directory where the main application is and a “lib” directory where the “Cloudformation Stack” is.

The “real code” is in setup-gitness-stack.ts where we create:

Required modifications?

You only need to set up “Secrets” in your repository with the names AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

After the initial deployment, you will need to modify the GITNESS_URL_BASE in https://github.com/Lock128/setup-gitness/blob/main/lib/setup-gitness-stack.ts#L104 to point it at the Load Balancer URL that has been set up for you.

Deployment

If you “fork” this repository and then push something to the “main” branch, a Github Action workflow will deploy this to your AWS account.

The deployed CloudFormation stack will contain everythign that you need:

The expected costs for this is roughly 50 USD / month.

Next steps from here

As mentioned above the code presented here is not production ready and it does not allow to use all of the functionalities of Gitness.

Things that you will need to consider / think of when using this code as a starting point:

  • What’s the backup schedule for the data on EFS?
  • What’s the required scaling policies?
  • Do you need a custom domain name?
  • Do we need to do security hardening – on the image, on the infrastructure, etc.?
  • Do we need to have multiple environments (development / production) for Gitness?
  • How to connect to this Git server from your VPC?
  • How to automate user creation or tie it to an existing AWS Identity Center?

As you can see, this is just a starting point of your journey to host your own Git Server on AWS 🙂

Please let me know if you have better ideas, suggestions or alternatives!

Views: 746

The state of CodeCatalyst in July 2024

I am personally using CodeCatalyst regularly for a lot of private projects, I also work a lot with other users of CodeCatalyst and I give feedback to the CodeCatalyst team regularly. In this post I look at the state of the tool in July 2024 and about how I make use of it on a regular basis.

A few more months in…

CodeCatalyst has been officially announced in december 2022 and reached GA in april 2023. Since then, it has been getting a lot of updates and changes, some of them you’ve potentially never had a look on.
In december 2023, major updates for enterprise customers were announced alongside other features like packages and Amazon Q integration functionalities.

CodeCatalyst Best New Updates in July 2024

Since last re:invent, CodeCatalyst has gradually increased the third party integrations with the option to have your source code stored in Gitlab, Github or Bitbucket. We have also seen the expansion of Custom Blueprints to CodeGeneration for repositories stored outside CodeCatalyst itself.

Just recently, we have also seen the possibility to have more than one space attached to a single IAM identity center, which allows further usage of CodeCatalyst for more enterprise customers.

CodeCatalyst also announced the possibility to expand packages usage to other providers than just npm – you are now also able to store maven based artifacts or OCI based images in packages.

Major updates to custom blueprints and additional blueprints allow and anble you to on the onside import source code into CodeCatalyst and on the other side to create a custom blueprint out of an existing project. This should make creating blueprints more accessible.

For a few months it has also been possible to include “approval gates” in CodeCatalyst workflows. This is a very limited functionality, but it still allows some important use cases.

Is CodeCatalyst ready for prime time?

It still depends.

While CodeCatalyst has drastically improved and matured over the last 12 months, there are still a few things that need to get better before I would 100% recommend you to use it.

Things that mainly concern me as of now: CI/CD capabilities and integration with AWS services.

The CI/CD capabilities are still limited and need to be improved to be more flexible and integrated. Approval rules need to be more sophisticated and allow some more specification.

If you already have CI/CD workflows or branch permissions set up in a tool of your choice, having “import” functionalities that translate existing Github Actions, Jenkins pipelines or Gitlab workflows into CodeCatalyst workflows is missing as well as the option to automatically set up branch permissions.

Other than that, CodeCatalyst is pretty much ready to be used for prime time and it has some functionalities that are outstanding and should be marketted more.

Next steps? What I think could come next…

The brave option

I still believe that that most underrated functionalitiy of CodeCatalyst is the Custom Blueprints functionality. If you’re living in a k8s world, Backstage has been leading, together with others, the field of “Internal Developer Portals” that empower developers perform actions quicker and more eficient in their day to day live. Especially Backstage starts with the possibility of scaffolding projects and generating code. However, Backstage does not allow you to keep track of changes to the relevant templates later.

Custom Blueprints – and also “existing Blueprints” do empower developers to do exactly the same thing.

Given CodeCatalyst has already been opening itself with other third party integrations like allowing a full Github, Gitlab and Bitbucket integration, I can see a potential of opening CodeCatalyst up even further.

With the given and already available marketplace in CodeCatalyst – that is not yet used very much – this could be opened up to allow other providers to add additional integrations, actions, blue prints.

Still the team would need to add additional functionalities like dashboards, widgets, … to make CodeCatalyst like an “Internal Developer Portal”.

What is unclear to me is whether AWS will be brave enough to perform another 1-2 years of investment into CodeCatalyst before it can become the central place for developers on AWS. I am also not sure AWS will finally go All-In on CodeCatalyst or if they will continue to invest into the existing Code* tools (CodeCommit/CodePipeline/CodeBuild/CodeArtifact).

The usual way for AWS developer tools

AWS will continue to invest half-focused and try to stay “on track” to help a huge customer base to achieve the simple things with CodeCatalyst. Integrations to other AWS services will be missed, the adoption rate will be small. With this kind of investment, AWS will have multiple solutions of Developer Tools (CodeCatalyst vs. CodeCommit/CodePipeline/CodeBuild/CodeArtifact) in the portfolio that both do not solve “all” problems and usecases but serve different customer bases.

What I think will happen

Give CodeCatalyst is build in different service teams we will see some teams heavily investing into making “their” part of the product successful (e.g. “Packages”, “CI/CD” or “Amazon Q in CodeCatalyst”). We will start seing these unique capabilities reach other parts of AWS services or potentially also other platforms. CodeCatalyst as a product will continue to exist but the different service teams will start to focus on where they can make more “money”. CodeCatalyst will not be able to deliver the promise it had when it was announced as the “central place for DevOps teams on AWS”. CodeCatalyst functionalities will be made available through the AWS console. With that, CodeCatalyst as “the product” that I was hoping for will cease to exist.

What do you think about my ideas and assumptions? Do you think I am wrong?

Drop me a comment or a note, I’d love to hear what your take on the future of CodeCatalyst is!

Views: 1411

Application Composer levels up a lot and adds amazing IDE integration capabilities

In this post we’re going to look at the new functionalities that have been added to Application Composer

In this post we’re going to look at the new functionalities that have been added to Application Composer by re:Invent 2023. After announcing the support of all CloudFormation resources earlier in the year, Application Composer now allows editing StepFunctions within the same user interface and – even cooler – announces the integration of an IDE plugin that allows developers to build serverless functions locally.

Application Composer as a serverless, rapid prototyping service adds additional capabilities to empower developers building serverless applications

Application Composer, that was originally announced last year at re:Invent 2022, has gotten a lot of major improvements thoughout 2023. As we are right at re:Invent 2023, its time to look back on which new capabilities have been added and how they influence building serverless applications using AppComposer.

Supporting all CloudFormation resouces

Already a few weeks ago the team announced that all over 1000 CloudFormation resources are now supported by AppComposer. This really gave a big update and make it simpler to build all kind of serverless applications. However, as this only alled AppComposer to expose the resources, this still requires the developer to know all required connections between the different resources. I personally would love to get more “supported” resources (just like L2 resources in CDK) to be made available as part of AppComposer. I would hope that this will be an additional functionality soon.

Integrating additional services

With the integration of the Stepfunctions Workflow Studio within the same interface, developers can now build and end to end application within composer before using the generated SAM or CDK templates to trigger the deployment. As a next step I think it would be great to also be able to define Eventbridge Rules & Pipes within the same interface.

Local development and IDE integration

AppComposer announced a Visual Studio Code integration that makes it possible to build and design serverless applications right from your IDE!

With this feature, you can visualize your serverless applications without being within your browser or the AWS console – start building, whereever you are and whenever you want!

I have not been able to try out this functionality yet, but especially the integration with sam connect that allows to also directly deploy the changes you made to your picture / template will make a big different in building applications using AppComposer.

Also I think we should not underestimate the possibility that this offers to vizualize existing CloudFormation templates through either the IDE plugin or the AWS Console. This will help to explain big and difficult already existing applications and empowers teams to have a fruitful conversation about changes they would like to implement in existing templates, as have a visualization makes the conversation easier.

What’s next for Application Composer? What are my wishes?

Already last year I have asked to integrate AppComposer in CodeCatalyst and I believe that this would be an awesome possibility to quickly start serverless projects. Application Composer today feels like a playground – to make the service more usable, it needs to have a “deployment” component that allows you to automate the lifecycle of your serverless application (including a full CI/CD pipeline).

I also last year asked for creation of CDK out of Application Composer – or even importing it – but instead of investing into that direction AWS recently announced the existance of the CDK Builder Tool – wouldn’t it be better to merge those initiatives together?

As already mentioned above, supporting additional “CDK-L2-like” patterns – or maybe the “Patterns” from serverlessland.com would be amazing – so users do not need to know how to set up IAM roles, connections between API Gateway and Lambda, … manually would make this a much more usable product!

What are your thoughts around the recent announcements of AppComposer? What are your experiences with it?

Views: 615

re:Capping re:Invent 2023 – Not everything that happens in Vegas should stay there! Let’s go and build!

In this article I will try to re:Cap a few of the announcements at re:Invent 2023 but also share my personal experiences and learnings that covers what I think that should be shared with the world…!

What happens in Vegas…

…should not always stay in Las Vegas! This year’s re:Invent has been another great experience for me and it was amazing to meet AWS enthutiasts from all of the world. I’ve learned a tons of stuff, saw a bunch of cool sessions and also experienced to be part of a big family. All the friendships that have been build in the past few days, the shared knowledge and experiences that have been shared have a big influecnce on myself and shape me.

The technical aspects of re:Invent

This year the technical aspects of re:Invent existed but where not as important to me as they used to be in my previous attendances. Of course AWS hd a bunch of important announcements – some of them bigger, some smaller. Renato has them written up at InfoQ and the AWS Newsblog has them covered too. Luc, the winner of this years “Now, Go Build” award 2023 has created a web application that helps you to read all of them and not miss a single one.

For me, there are a few that stand out:

Of course, there where a bunch of other announcement, minor and smaller ones, but these are the ones that I have remembered and thus they are meaningful to me. Now let’s move over to the more important aspects of re:Invent!

The community aspects of re:Invent

re:Invent 2023 has been once more a gathering of the AWS Community in one place and it has brought a lot of us together to talk, laugh and align. Not everyone was able to join us due to different reasons – but I am sure that you have felt the power of the community throughtout the week by following us on the different social channels.

Being part of the AWS Heros

As I posted last year, going to re:Invent means meeting with friends and getting together. Being an AWS Hero, made it more intense than before: We feel community out of our heart and that’s what makes us strong. Wherever I was in Las Vegas, I saw a fellow Hero.

We all have super powers and our powers are different. One of my super-powers is connecting people – and I hope that I was able to show this in the last few days.

Others have other powers – a few of use were able to present one of their talks – Anahit with her spciality around MSK, Anurag around data patterns and Ran on Lambda Power Tools. Others are great listeners and others have the vision of how things need to or should look in a few years – it was great to see everyones powers in one place and I know that combining them we can incluence to make things better!

Thank you, Taylor and the rest of the team, for creating this group and bringing us together again!

Working with Builders, User Group leaders and others from the community

The AWS Community consists of so much more than the Heros. Thank you, dear Community Builders – lead my Jason and the team – for being an unbelievable source of power throughtout the week. Your entuthiasm, your great ideas and your dedication are what makes us stronger. I’ve been reading a lot of the posts from Builders around the globe that were not able to make it to Las Vegas and it is energizing to see that.

The User Group Leaders that we have world wide on the other side help to thrive the AWS Community across the whole yearand bring us together regularly — to learn, to play or to share knowledge. Thank you all, for helping us to shape where the community goes and for making the community successful. I was glad to be able to meet a lot of you and share my experiences as welll as listening to your experiences.

I had the great pleasure to get the whole team of core contributors of the Speakers Directory together and we were able to present our project as well as take a picture of all of use 🙂

We are going to continue our investment and will help user group leaders to find speakers through our tool!

Working with AWS employees

This year, I’ve joined the club of many other Heros that go to sessions where they can meet AWS service team members that they have worked with before 🙂

I attended a few CodeCatalyst sessions to meet the team that I’ve been working with for more than 12 months “live and in person” and loved to see the energy and innovation live on stage – but I also attended other sessions just to say HI to certain speakers.

Employees at AWS are smart and can often tell you the perspective of WHY something has been build and it’s great to know some more background of a new feature. Thank you all for spending time with me and sharing your thoughts and passion with me!

To those AWS employees in the community and DevRel team – another big THANKS for making the event unforgettable with all of your dedication and support – I love spending time with you and creating new ideas on how to make the AWS community stronger and more engaging than ever before!

A look ahead…

As I try to use my time on the flight to put my head around what I am taking away from the last few days and from re:Invent 2023, I’m still digesting, as many others, what we have all learned and heard.

A few key take-aways:

  • AWS doesn’t feel “secure” anymore to be a market leader
  • innovation at AWS is coming (Q), but it’s still early stages
  • AWS keeps listening to their customers (see the DB2 RDS announcement and the StepFunction HTTP Integration)
  • Community Sessions (COM or DEV track) are the ones to attend at re:invent, or sessions that are AWS + a customer (level 300/400)

What I’m considering to do in the next 3 months

First of all I’m planning to cover the CodeCatalyst announcements at my YouTube channel to explain the impact of the new features to interested enterprise customers.

I’m also looking at trying out a lot of the cool things that have been announced in our AWS Speakers Directory Project, besides hosting multiple User Group meetups of the AWS UserGroup Bergstrasse.

What I’m considering to do in 2024

Of course I will continue my engagement in the AWS Förderverein DACH – we are planning another AWS Community Day in Munich next year!

I also plan to continue my work with the the CodeCatalyst team to shape the product – please let me know if YOU have input on what thte next important steps are.

I would love to work with the AWS team to, for 2024 at re:Invent organize another pre:Invent Community Hike and to talk about the possibility of hosting a complete track at re:Invent where Community Members join forces with AWS employees. I listened to a session (Ran Isenberg and Heitor Lessa) and that was a very powerful message.

Last but not least, I would like to help community members to grow and shape their careers in Cloud – if you need help or have questions, do not hesitate to reach out for questions, I’m happy to help or to connect you with someone that can help!

Thank you for reading until this point, if you have any feedback, let me know!

Views: 389

Amazon CodeCatalyst’s packages support – a glimpse at what’s to come for artifact management

In this post you will get a short introduction and my personal assessment on how the new Packages component in CodeCatalyst can help you to set up your complete SDLC in Amazon CodeCatalyst.

Only npm supported – a early launch to show what we can expect going forward and to get feedback from users?

With the initial launch Amazon CodeCatalyst allows to manage npm package repositories inside CodeCatalyst. When seing this for the first time I was a bit concerned that this decision – to initially launch with only npm support – will not help a lot, as I expect that organizations would need to store other type of artifacts (jar files, containers, python paackages, …). But I think that at least for Cloud Native Projects that use typescript this will solve the problem of storing artifacts and accessing them natively within CodeCatalyst. Let’s look at how they can be used today.

Using packages repositories in CodeCatalyst

Today you can set up package repositories in CodeCatalyst and connect to upstream (public or private) repositories. You can also change the sort order and the upstream order of the package repositories. The documentation covers the different possibilities to set up repositories very well. You can access the package repositories from your local machine by setting up the connection locally

npm config set registry https://packages.region.codecatalyst.aws/npm/space-name/proj-name/repo-name/

After setting up multiple repositories you can set the order of retrieval and usage. You can also set it up as a “pass-through” repository and allow access to public repositories.

Using packages in CodeCatalyst

Packages in CodeCatalyst are integrated with workflows. You can read or store packages with native actions or by setting up the npm repository manually.

You can also read and publish packages from other systems by using Personal Access Tokens (PAT). The documentation outlines currently supported client commands.

The CodeCatalyst workflows will by default use the set up repositories that you have set up within CodeCatalyst.

What I think we need next in packages

With the launch of the packages component in CodeCatalyst AWS has added a definately missing functionality in CodeCatalyst. Users are not able to store artifacts and share them within a space which makes it possible to create, re-use and deploy immutable artifacts natively within the tool.

This is a much needed functionality, but as I already mentioned, I believe that limiting this to npm packages only limits the use cases for it. I would have expected npm, python and docker support at launch and was a bit disappointed that this was not included.

I do also believe that the “packages” functionality should be better integrated and allow further configuration options, especially when loading/reading packages from upstream repositories – e.g. being able to limit to only certain licences or ensuring that packages included are still “supported” (and regularly updated) or security-scanned.

These type of functionalities would have made the new option “meaningful” and would have empowered developers to build better software too.

I can imagine that we will see python, java and dockersupport pretty soon due to these reasons and it would be great to also support “internal” repositories as “upstream” repositories.

How do YOU think that this new functionality will help you to adapt CodeCatalyst?
Please drop me a message on socials or an E-Mail!

Views: 632

Amazon CodeCatalyst’s integration with the IAM identity center allows SSO with your own IdP

In this post you will learn how the new integration of the IAM Identity center with Amazon CodeCatalyst allows to use your own IdP as single source of truth of users for CodeCatalyst spaces. We will look at how you can connect a space to an IdP and on how this can be used to set up certain permissions based on your IdP roles.

Single Sign On (SSO) using IAM Identity Center empowers CodeCatalyst users to use their own IdP to provision user accounts in CodeCatalyst

With the new integration for Amazon CodeCatalyst into the AWS Identity Center organizations can now connect to their own existing IdP (AzureAD, Ping, Okta, …) going through the AWS Identity Center.

Benefits of the new integration

With this new integration users and organizations go to the Space settings in CodeCatalyst and choose to integrate with an existing instance of the IAM Identity Center.

This also connects potentialy to any other IdP that uses the IAM Itentity Center as a proxy. Administrators will only need to manage roles and permissions in on central place. The roles from the IdP can then be mapped to certain team roles in Amazon CodeCatalyst.

What I don’ t like about the new integration

There are two things that I didn’t like when looking at this integration:

Why do we need to go through the IAM Identity Center and cannot allow direct SAML or OIDC integration for a specific space? With the “need” to have an IAM Identity center organizations also need to directly connect to an existing AWS Account to manage connectivity – and this once again brings back a “tooling” account that initially I was hoping to replace with CodeCatalyst.

The second thing that I missed was the possibility to automatically map existing roles to groups in Amazon CodeCatalyst – today this is a – one time – manually effort that needs to be conducted. If you need to do that only once for your organization that isn’t a problem, but if you decide to have multiple spaces for your organization and need to perform this multiple times, then this can be very time consuming.

Summary

I believe that this change will have a very big impact on the adoption for CodeCatalyst as administrators and security teams in organizations will make it easier to allow the usage now that there is a central place to manage identities. And with the possibility to actually try out CodeCatalyst in an enterprise world, I hope that we will also see more adoption of CodeCatalyst and with that also a more streamlined and better communicated roadmap of CodeCatalyst.
If you would like to learn more on how to activate this, please refer to the documentation.

Views: 233

Hey, I’m going to San Francisco and Vegas to meet friends at re:Invent 2023

In this post I’m writing about how getting to re:Invent 2023 was and about how the last year has made a big impact on my personality and my efforts in the AWS community.

Thinking it couldn’t get better last year, 2023 changed me once more and leveled me up

While going to re:Invent 2022 I wrote down my thoughts and went home from re:Invent expecting things could not get better for me. But this year has prooven my expectations wrong, as the last 10 months have been life-changing in a lot of different aspects.

Blogging & Starting a YouTube channel – winning 1.5 hackathons and starting a project

When I reached my home town after last years re:Invent I knew that this community is what makes me happy and gives me back so much that I also would like to give something back. And so, I formed a few ideas in my head and started a few initiatives that I would like to share with you today:

Blogging & Writing

I have been blogging both on dev.to and on my personal blog – writing about topics and things that I do or did on a regular basis, trying to share experiences of day-to-day things and building experiences.

The most important thing for me he is to always be transarent and authentic – speaking out on things that I liked and challenges that I’ve faced.

For you reading this – thank you, for being part of my journey, for giving me feedback and for being you! Please let me know if you have any feedback on what I should change or do better.

Kicking off a YouTube channel

The last three years of my career have been focused on building secure and reliable CI/CD pipelines! But hey, there is so much more to learn. As part of my travel back to my home town after re:Invent, I had the thought to share my experiences and the experiences of other builders on a “podcast-like” YouTube channel which is today known as @cicdonaws.

Learning on how to produce a YouTube video to how to set up a good lightning and sound, preparing a script and creating better thumbnails – a lot of learnings, experiences and hours went into the videos that I have been producing.

This only kicked off because of all of the great builders that joined me on the show and shared their projects, learnings and experiences with me. Thank you all, every single one, for your time, dedication and experiences. Keep up sharing your knowledge, as this is how we can all grow!

Winning 1.5 hackathons and making a small project something big

Back in march Lily announced a, Community Builders-internal hackathon where it was possible to win some prices – and I thought I had a great idea on what to build that actually fit into the “rules” of the hackathon. And this is how the project started that has since then used up countless of hours of my after-work evenings and weekends. Millions of Slack messages have gone from Germany to US, from US and Germany to Africa and back – we kicked of a real project with our Speakers Directory that is finally ready for prime time! After becoming second in the first Hackathon, we also submitted the same project with a few tweaks for a 2nd hackathon powered by Hashnode and Amplify and this also brought us “honourable mentions” in the list of top 10.

We’re ready now to get more of you to add your talks and events! Please sign up and give us feedback, we are very proud of what we have produced so far and eager to get all of you on board!

Thanks to Danielle, Julian, Matt, Raphal, Baimam for actively working with me on this fun project – and thanks to everyone else that has supported us through 2023 to make this a bigger thing!

Organizing a community day – the EU (more technical) edition of re:Invent

Another piece last year has been organizing the AWS Community Day DACH that happened in Munich this year – it was fascinating to be part of the team that brought more than 500 AWS nerds together, including over 25 AWS Heroes and 30 Community Builers!

Thank you all for attending – and thanks for the sponsors for making this possible!

We’re up for doing this again next year – further details to be announced soon! Looking forward to see you there!

Hey, hiking before re:Invent is what you need!

And then, there is this “secret thing”: Ever since I went to re:Invent I used the sunday before re:Invent to go out “hiking” somewhere close to Las Vegas. The first two visits together with colleagues were three persons – a pretty cool small group.

Last year, they did not go with me to re:Invent, so I asked Community Builders to join me – at the end, we were nine persons from I think 8 different countries hiking together from 10 am till 5 pm – we had a lot of fun, so I decided: Let’s do this again!

Somehow, this became a bigger thing – for this years hike, we have over 50 Community Builders, Heros and UG Leaders signed up. The group became so big that the AWS Community team is helping us this year by sponsoring the transportation.

Thank you all, that you are supporting this thing – and thanks to everyone participating, I am sure this will be an amazing experience for everyone!

How and Why what I do for the community changed

Since June I can name myself an AWS Devtools Hero – which honors me a lot but also drove a bit on how and what I do for the AWS community. My work has become a little bit less “public” – a lot of my hours that I have “free” went into talking to AWS Service Teams, into mentoring other builders and helping them grow. This has lead to less new videos on YouTube and less blog posts – but instead, I’ve helped others to grow and to invest into the community – supporting the “re-start” of the AWS UG Frankfurt, the re-start of the User Groups Mainz, Karlsruhe and Heilbronn and overall making other builders “grow”. This makes me happy. Thank you all, that you are part of my journey.

Making new friends

This year has also shown to me how important relationships are and how much you can make friends by supporting each other. Friends, I’ve never met in person (Matt), friends I’ve finally met in September (Ran) and others like Markus, Thorsten, Philipp and Raphael that are part of most of my days and that I write 100s of Slack messages per week.

This is what community means! Making friends, learning, growing. Thank you, for being you!

What’s next for me?

Changing my role at work

I’ve recently moved to the global Platform Architecture team here at FICO and that gives me a lot of new possibility and learning opportunities.

The platform that FICO is building is an important one and helps organizations around the globe to take better decisions – I’m proud to be part of the team and looking forward to make this bigger than it is today.

Community Day DACH 2024

We’re already planning next year. Stay tuned for some announcements…soon!

Supporting and mentoring builders around the globe

I’m up for helping other builders grow – if you’re intersted to collaborate with me or to learn from me, reach out to me and we can talk!

If you have any CI/CD or CodeCatalyst specific topics that you would like to talk about on my channel – reach out to me!

If you need feedback, help or advice on anything (e.g. a blog post or anything else) – let me know and reach out!

Looking forward to an amazing re:Invent 2023 with a lot of friends in Las Vegas!

Have a great week – don’t be a stranger and say HI if you see me around!

Views: 632

Amazon Codecatalyst reaches “GA” status and becomes available for general use

The new service announced by Amazon in Las Vegas at re:Invent 2022 which is an integrated DevOps service to empower development teams to develop and deliver software faster finally reaches the “general availability” status. As I have previously outlined, this achievement is very important for Amazon and the CodeCatalyst team. Congratulations to the team for reaching this goal, which I can imagine is not an easy step for this product. The tool touches a lot of very sensitive parts of a software project and I can imagine the security standards being really high. 

A hugh achievement – thank you everyone in the team for investing into CodeCatalyst and for listening as closely to the customer feedback as you are!

What changes did get implemented for GA?

As part of the GA release we see a lot of minor improvements in the User Interface and color changes. In the last weeks, we have seen a few “bigger” changes – like the possibility to use Dev Environments for Github based projects. We also got “graviton based” execution environments for CI/CD workflows which, according to AWS, should reduce our costs.

It is still hard to track down all of the changes in CodeCatalyst, as there is – to my knowledge – no public or semi-public roadmap. This is one of the things that I’d love to see, as for an integrated service that is at the core of the Developer Experience for teams, any minor change can either improve or destroy the “usage experience”. If you as a team invest into adoption a new tool like CodeCatalyst, they will need to know how changes in workflow, features or user interface can influence their day-to-day activities. Let’s see, maybe the team can share “something” like a “changelog” with us (or even an RFC process like Amplify or AppSync)?

Reached “GA” – so who can start using it now?

As of today CodeCatalyst is only availble in US regions and this means that it can be adopted mainly by US enterprise customers. CodeCatalyst already gives you the possibility to set up different Spaces for your account and within a space you can manage multiple projects. So in theory, CodeCatalyst is “ready to be used” by everyone. 

Practically speaking, it is easier to adapt the service for new projects than for existing projects , as there is no real “import” functionality. Yes, you can integrate existing Github projects, but that only integrates the source code. Unfortunately that does not make all of the “cool” things available right from the start of integrating the source: existing workflows (CI/CD pipelines) are lost and need to be re-build, issues/tickets are not imported into CodeCatalyst (tho they can be made available through the JIRA integration). 

I have been regularly using CodeCatalyst (both for imported and “new” projects) – and I really think that the tool already works very well. 

The “killer feature” that I see for new projects are the “blue prints” which essentially get you started within minutes, e.g. to deploy a SPA application, or to have a “true” CI/CD pipeline for a full stack application following the DPRA. 

Right now I would recommend using CodeCatalyst for any new project that you start to start building out your workflows and best practices.

So what do I still need to recommend CodeCatalyst for existing projects?

There are a few things that I have already been writing about:

  • “Import” of existing CI/CD workflows (e.g. Github actions, CDK Pipelines or CodePipelines)
  • Fully import projects
    • existing issues from Github or JIRA
    • Git-based projects including the history
  • Tighter security settings and permissions
    • Fine granular roles to allow or forbid access to specific parts of a project
    • Options to allow or forbid execution of workflows (or to deployments)
  • Additional workflow options
    • Manual approvals are very high on my wish list
    • Integration of other AWS services natively

A question for the readers: What do YOU think that you need to adopt CodeCatalyst?

A big question for the CodeCatalyst team – HOW MANY AWS TEAMS ARE USING CODECATALYST FOR PRODUCTION DEPLOYMENTS TODAY?

Where do I see the potential for CodeCatalyst?

CodeCatalyst is a big bet by AWS. There is a big potential that can really improve the life of development teams and these are the main things that I believe that can out-grow other existing solutions:

  • Integration of AWS Services / deployments metrics
    • the true integration with AWS APIs
    • Integration into “post-deployment” verifications (e.g. auto roll-back after failed CloudWatch metrics)
  • “At-hand” developer support to improve efficiency
    • with CodeWhisperer (who recently reached GA) AWS already aims to support developers during the development phase, but with CodeCatalyst AWS can take this to the next level:
    • AI support during Pull Request Reviews (or automated approvals for PRs – e.g. by including CodeGuru, etc., automated merges, etc.)
    • AI support during workflow executions (when to approve, when to deloy, when to promote, etc.)
    • With improvement proposals for workflows if the “AI model” recognizes patterns (in issue workflows or CI/CD workflows)
  • With automated improvements for existing projects based on blue prints
    • Best practices change – and so blue prints change – and if the CodeCatalyst team can automatically apply them for existing projects, customers will benefit from it

And last but not least:

I trully believe that every software project should start with a CI/CD pipeline – and with the Blue Prints including the CI/CD workflow that follows DPRA and other AWS best practices, we can trully make this possible: Empower developers to deliver their software projects in minutes right after starting their project.

Do you see the potential in CodeCatalyst? If you do not see any potential in the tool – why not?

Views: 1791