As Amazon CodeCatalyst is still in Preview, it has only limited integration possibilities with other AWS services or external tools. Sending notifications from a Workflow execution is something that I believe is critical for a CI/CD system – and as I focus on CI/CD at the moment I’ll focus on the notifications on Workflows in this article.
What kind of notifications do I need or expect?
As a user of a CI/CD and Workflow tool there are different levels of notifications that I would like to receive:
Start / End and Status of Workflow execution
State / Stage transitions (for longer running workflows)
Approvals (if required)
In addition to that, based on the context of the notification I would like to get context-specific information:
a) For the “Start” event I would like to know who or which trigger started the workflow, which branch and version it is running on, which project and workflow has been triggered. If possible getting the expected execution time / finish time would be good b) For the “End” event I would like to know how long the execution took and if it was successful or not. I would also like to know if artifacts have been created or if deployments have been done. If the “End” is because of a failure, I would love to know the failure reason (e.g. tests failed, deployment failed, …) c) For the state transitions I’d love to know the “time since started” and “expected completion time”. I would also like to, obviously, know the state that has been completed and the one that will now be started. d) For approvals I’d love to be able to get the information about the approval ask and all required information (commit Id, branch) to do the approval
What does CodeCatalyst Support today?
Right now CodeCatalyst allows to set up notifications to Slack. Please see details on how to set this up here. This notifications are also minimal right now:
In Slack this looks like this:
How can I enhance the notification possibilities?
Luckily one of the “core actions” is the possibility to trigger a Lambda function and this is what we are going to use here to be able to trigger advanced notifications using Amazon SNS. In our example we are going to use this to send an eMail to a specific address, but you can also use any other destinations supported by SNS like SMS or AWS ChatBot.
Setting up pre-requisites
Unfortunately we will need to set up an SNS topic and a Lambda function in a dedicated AWS account in order to use these advanced notifications. This means that we are “breaking” the concept of CodeCatalyst not requiring access to the AWS Console, but this is the only way that I found so far to be able to send additional notifications.
Ideally we would be setting up the SNS topic and the lambda function using CDK, but that increases the complexity of the workflow and of the setup and because of that I’m not including that in this blog post.
Setting up the SNS topic
Please create a SNS topic following the AWS documentation through the console. We assume the topic to be in “eu-central-1” and the name to be “codecatalyst-workflow-topic“.
You can follow this blog post to manually set up the lambda function through the AWS console, please ensure to give the Lambda functions permissions to use the SNS topic. The required code using Python will look like this:
Obviously the same can be achieved using Typscript, Go or any other supported function. Please adjust the topic_arn to match the topic that you just created. After creation this Lambda function will now have an ARN which should look similar to this: arn:aws:lambda:eu-central-1:<accountId>:function:send-sns-notification-python
We will need this ARN when setting up the notification in our Workflow.
Integration into the workflow
Integrating this Lambda function into a workflow is easy:
As you can see, we are integrating an “aws/lambda-invoke@v1” action which then points to the lambda function that we just created.
In the “RequestPayload” we are passing a few information to the Lambda function which will then be passed to the SNS topic as part of the message. This is how the message will look when received as eMail:
Missing information and next steps for enhanced notifications
As you can see we are able to send notifications from CodeCatalyst to multiple targets, including eMail with this option.
What we are missing is – and I am not sure if thats possible or not – is all of the “metadata” of the workflow execution like:
Workflow-Name
State-Name
Project Name and additional information
In the documentation I was not able to find out the available environment variables about these information…. If you do have any ideas on how to access this metadata, please let me know!
In the last weeks – or already months – I’ve been working together with Christian, also an AWS Community Builder, on our project named “Football Match Center”. Christian has already been writing a lot about our project on LinkedIn:
Today, I want to put the attention on our chosen framework for the UI and the way that we are connecting from the UI to the backend. Our backend in this project is a GraphQL API endpoint hosted on AWS AppSync.
Building our UI in Flutter
Since last year Amplify Flutter includes support for Web and Desktop. As we are looking to reach users both on mobile as also on the desktop, choosing a cross-platform development tool like Flutter seemed to be an obvious choice. Christian and I are a small team, and we want to focus on building a simple UI quickly without the need to implement for multiple platforms and Flutter allows exactly that.
Flutter provides easily extendable widgets that can be used on all major platforms.
Connecting to our GraphQL backend
Our project is not based on an Amplify backend, but on AWS infrastructure written in AWS CDK. This made it rather difficult to use the Amplify Flutter SDK as most of the documentations and blog posts expect you to connect the Amplify SDK with an Amplify backend (which can then include a GraphQL API).
But that’s not only what made it difficult – I also had very little experience with Amplify or the Amplify SDK when starting to work on the connection.
Using the Flutter SDK for Amplify we will be connecting to our Cognito instance for Authentication and to our existing GraphQL endpoint. In this post I am going to look at the GraphQL connection and not on the integration of Cognito as an authentication endpoint.
Setting up Amplify SDK for Flutter can be done through the amplify cli if you are starting a new project.
This will then also create the required amplifyconfiguration.dart and some example code through amplify init.
You can then set up the Amplify SDK for Flutter from within your main widget using this code:
import 'package:amplify_flutter/amplify_flutter.dart';
import 'package:amplify_api/amplify_api.dart';
import 'amplifyconfiguration.dart';
import 'models/ModelProvider.dart';
….
Future<void> _configureAmplify() async {
final api = AmplifyAPI(modelProvider: ModelProvider.instance);
await Amplify.addPlugin(api);
await Amplify.configure(amplifyconfig);
try {
await Amplify.configure(amplifyconfig);
} on AmplifyAlreadyConfiguredException {
safePrint(
'Tried to reconfigure Amplify; this can occur when your app restarts on Android.');
}
}
While this looks easy when reading the documentation (and a lot of very good blog posts), this was rather difficult for me as I was not able to use the amplify init command. Finding out the structure of the “amplifyconfiguration.dart” and the implementation for the “ModelProvider” were my main challenges.
Lately, the related documentation has been updated and it is now easier to work with existing resources.
The Amplify Configuration file
The Amplify Configuration (amplifyconfiguration.dart) configures all of the required Amplify Plugins. In our implementation we started with the GraphQL backend:
This tells the Amplify SDK to talk to a specific API endpoint when the “Amplify.API” is invoked. As far as I understand this Github issue, right now only one API can be queried from a specific Amplify instance.
When using the apiKey to do the authentication with the API, we will need to regularly update the Flutter application as the default API expires after 7 days.
This documentation was not available when we started to work on the project and I have the suspicion that Salih made this happen 🙂 (if not, still THANKS for the help you gave me! 🙂)
The ModelProvider
The ModelProvider should be a generated file, which you can generate from an existing GraphQL API. If you are using a schema that is not managed by Amplify, you will need to use “amplify codegen” based on an existing schema file.
The command expects a schema.graphql to be available in the “root” folder of the Amplify Flutter project. If you execute “amplify codegen models”, required Dart files will be generated in the “lib/models” directory.
The result should be a file similar to this one:
import 'package:amplify_core/amplify_core.dart';
import 'Match.dart';
import 'PaginatedMatches.dart';
import 'PaginatedTeams.dart';
import 'Team.dart';
export 'Match.dart';
export 'PaginatedMatches.dart';
export 'PaginatedTeams.dart';
export 'Team.dart';
class ModelProvider implements ModelProviderInterface {
@override
String version = "4ba35f5f4a47ee16223f0e1f4adace8d";
@override
List<ModelSchema> modelSchemas = [Match.schema, PaginatedMatches.schema, PaginatedTeams.schema, Team.schema];
static final ModelProvider _instance = ModelProvider();
@override
List<ModelSchema> customTypeSchemas = [];
static ModelProvider get instance => _instance;
ModelType getModelTypeByModelName(String modelName) {
switch(modelName) {
case "Match":
return Match.classType;
case "PaginatedMatches":
return PaginatedMatches.classType;
case "PaginatedTeams":
return PaginatedTeams.classType;
case "Team":
return Team.classType;
default:
throw Exception("Failed to find model in model provider for model name: " + modelName);
}
}
}
Querying our GraphQL API
Now that we have been able to connect to our GraphQL AWS AppSync endpoint, we can start querying data.
Luckily, the preparations we made and the Amplify for Flutter SDK provides convenience methods that returned typed data structures that we can directly interact or work with.
You only need to write the GraphQL query that you are interested in and you can directly read data from the endpoint. In my example below I’m creating a Flutter Widget out of the returned elements and then I am adding them to a list of Widgets that I can display in a Column Widget:
Future<List<TeamWidget>> _getMatchesByCountry(String country) async {
List<TeamWidget> teamsWidgetList = [];
try {
String graphQLDocument = '''query ListTeams {
getTeamsByCountry(country: "${country}") {
nextToken
teams {
PK
PrimaryColor
SK
SecondaryColor
TeamName
}
}
}''';
var operation = Amplify.API
.query(request: GraphQLRequest<String>(document: graphQLDocument));
var response = await operation.response;
var data = response.data;
if (data != null) {
Map<String, dynamic> userMap = jsonDecode(data);
List<dynamic> matches = userMap["getTeamsByCountry"]["teams"];
matches.forEach((element) {
if (element != null) {
if (element["id"] == null) {
element["id"] = "rnd-id";
}
var match = Team.fromJson(element);
teamsWidgetList.add(TeamWidget(match));
}
});
}
} on ApiException catch (e) {
print('Query failed: $e');
}
return teamsWidgetList;
}
Just today, we have merged a feature that adds a “subscription” to our AppSync endpoint – as as next step we plan to integrate this within the Amplify Flutter Application which will then allow us to implement notifications to the end users. Unfortunately, the Amplify SDK for Flutter does not yet support in-app messaging as it does for Javascript.
What YOU learned – and what I learned
Through this blog post you have learned how to connect an Flutter application with Amplify using the Flutter SDK for Amplify. You have also got to know our project, the “Football Match Center” – and you’ve seen some code to make your start easier when talking to a GraphQL (AppSync) backend.
I have learned to work with the Amplify for Flutter SDK and also how code generators can help you to speed up your implementation. I’ve also gained experiences in accessing data from AppSync and on working with the returned data in Flutter.
Unfortunately, I have also found out that using the Flutter SDK for Amplify I can right now not implement the planned in-app notifications that Christian and I wanted to build for our Football Match Center to notify users about upcoming or recently completed games.
At re:Invent 2022 AWS announced Amazon CodeCatalyst and as you might have read on my blog or seen on my YouTube Channel I have been playing around with the service a lot. A few days ago, Brian asked me a few interesting questions, one of them being:
What’s the diff between CodeCatalyst and AppComposer?
Lately we had a Community Builders session with the Amazon CodeCatalyst team and similar questions came up in regards to comparing CodeCatalyst with other, already existing services.
And to be honest, the amount of AWS services that are related to building, managing or deploying software projects on AWS has grown a lot in the last years and it gets difficult to keep an overview of how these services play together and which tool has which functionality.
In this post we are aiming to compare and place CodeCatalyst in relation to other (new or already existing) AWS Services. We are also going to look at missing functionalities that are currently available in other services but not in CodeCatalyst.
Please be aware that these are all our personal opinions and based on our own understanding – some of it being assumptions.
This post was Co-Authored with AWS Community Hero Brian Tarbox – Thanks for your support!
AWS Services that we are going to compare CodeCatalyst with:
Amplify was released at re:Invent 2018 and has since then been improved gradually.
Amplify is a complete solution that lets frontend web and mobile developers easily build, ship, and host full-stack applications on AWS, with the flexibility to leverage the breadth of AWS services as use cases evolve.
With that AWS positions Amplify as a service that is able to reduce the heavy lifting on web or mobile developers that want to get started on AWS. AWS has extended Amplify into being a service that offers nearly all building blocks required as part of your SDLC process. It does not offer source code repositories, but CI/CD capabilities. You are able to configure the CI/CD pipeline and also provide your own build images. With the release of Amplify Studio in 2021 AWS extended the capabilities to include a “No-Code/Low-Code” capability that allows rapid-prototyping for web and mobile applications. The target audience for Amplify are Front-End and Mobile developers with no to minimal experience on AWS.
Application Composer
This is a new AWS service announced at re:Invent 2022 mainly focused on “rapid prototyping” helping you to quickly “paint” serverless applications – build our your architecture out with visualizations, Application Composer will create the required “starting code” (Cloudformation, but also Lambda code) in the background. As output you get a project in code that you can then commit to a Git repository or deploy out to AWS. Application composer enables Serverless developers to quickly prototype serverless applications and convert them into code that can then be used as a starting point for your project. Application composer does not provide Source Code management or CI/CD capabilities.
The service, which reached GA on March 8th of 2023, points at developers starting new serverless projects that quickly want to get both an architecture diagram as well as a starting point for further developments.
App Runner
This is a AWS service announced in 2021 and it can be used to build, deploy and run web applications based on containerized workloads. It allows you to stay focused on your application with the service taking responsibility to provision and host your application. It also takes care of creating a container from your source code. You can connect App Runner either to your source code management system or to a container registry.
Beanstalk
This is one of the “ancient” AWS services – it was announced in 2011 and has since then been around. In the community I have more than once heard that “Beanstalk is dead” and not being actively developed anymore, but still – it works and can be used to provision your web applications. At the same time, you will still be able to access the infrastructure that is required to host your service. The “message” is similar to App Runner – it helps developers to focus on writing business code and ignore the deployment strategy. Beanstalk supports Java, .NET, PHP, Node.js, Python, Ruby, Go and Docker web applications. In order to use Beanstalk, you will need to upload a source bundle – it is not possible to connect beanstalk automatically to a Git repository, but you can update the source bundle automatically using APIs.
We treat these services in one group as they belong together from a strategic point of view. They have been around for a few years and the teams that built these are now involved in CodeCatalyst. CodeCatalyst partly uses them “under the hood”. CodeCommit is a managed git hosting, CodeBuild is a managed “build” system, CodeStar is a “project management” tool. CodePipeline allows combining multiple CodeBuild steps to form a pipeline. CDK Pipelines integrate with CodePipeline today. With CodeArtifact users are able to store artifacts and software packages.
All of these services are tied to a specific AWS Account and live within the AWS Console. This has forced organizations and AWS customers to create “toolchain accounts” that centrally host these services. These tools might be considered as building blocks rather than a full solution.
CodeCatalyst
As we are comparing the other services with CodeCatalyst, we also need to define what CodeCatalyst is: a new AWS service announced at re:Invent 2022 that will cover the full lifecycle of product development on AWS, starting from the source up to the deployment part. It is an “All-in-one” solution to help you build software on AWS efficiently. You can manage your planning and issue tracking in it as well as your source code and your CI/CD workflows. I have a few introduction videos recorded available on YouTube. CodeCatalyst lives “off” the AWS Console and this means that you do not need to be logged in to use it – and it can access multiple AWS accounts by an integrated authorization process.
Proton
This is a AWS service announced in 2020 – and AWS describes Proton as a service to allow central teams to build and provide central infrastructure components that can easily be shared with users while at the same time maintaining the integrity of the deployed infrastructure. With that, the tool is focused on infrastructure provisioning (=deployment) pipelines. Proton allows the central “platform team” to provide templates to be used by application teams – with only minor changes or configurations to it.
Which problem(s) does CodeCatalyst address?
CodeCatalyst addresses the need of developers or of development teams that need to cover all parts of the product life cycle or parts of it with a tool natively built on AWS. It can be used for issue management and planning as well as source code management. It has natively built CI/CD capabilities with workflows for Continuous Integration and Deployment. CodeCatalyst offers an opinionated solution for addressing software development best practices on AWS. It also allows online-editing of source code with the Dev Environments and supports the management with reports on resources and workflows managed as part of CodeCatalyst.With Blue Prints it allows developers to quickly start a new project and reduces the time to get a new project started. It can be seen as an opinionated approach to development.
So, how does CodeCatalyst relate to the other services?
Out of the six services we looked at, a few can at first glance not compete or be compared with CodeCatalyst as they target a different audience or address different problems as CodeCatalyst:
Proton – does not help with building or deploying code, it is targeted towards “composing” an application from various pieces. As such, it might be part of a solution but not the whole solution
Application Composer – while this service can be used to do a rapid-prototyping for serverless architectures, it does not allow source code management or deployment of the built architecture. I hope that we will see Application Composer as a new option for starting off a new project in CodeCatalyst going forward
Beanstalk – is not a “developer focused” tool as it comes with pre-build environments and CI/CD pipelines and expects you to manage the source code externally
Based on this, the services we want to look at in more details are:
While Amplify allows to build CI/CD pipelines and manage deployments for both Front-End and Back-End components of an application, the pipelines and deployments are limited to the services supported by Amplify and the capabilities of the automatically generated CI/CD pipeline. There is not much flexibility to adjust the pipelines. In addition to that, Amplify does not allow you to store your source code or to manage your software project. It has no build-in issue management or tracking system.
With Amplify Studio and the corresponding tutorials you get the possibility to quickly get started on specific use cases. This is not as flexible as the CodeCatalyst Blue Prints but gets you started pretty quickly. Amplify Studio is awesome as a “low-code”, getting you started tool – it allows you to quickly build full-stack applications through a User Interface and for that use case it is definitely better than CodeCatalyst. At the Berlin Summit in 2022 I attended a Live Demo of Rene Brandle and was amazed by the functionalities.
Amplify Studio lives “outside” the AWS Console in the same way as CodeCatalyst and it also requires an AWS account to be connected for deployments. Each Amplify project can be connected to one AWS account. This is more flexible in CodeCatalyst.
Still, Amplify misses a lot of things that are required for an end-to-end “DevOps” tool to manage all processes and requirements of an agile software development project.
Comparing CodeCatalyst to the Code* services (CodePipeline / CodeCommit / CodeBuild / CodeStar / CodeArtifact) feels a bit like comparing a Tesla Model 3 with Karl Benz’ Patent-Motorwagen 🙂
The Code* services feel complex to use, although they provide similar functionality than CodeCatalyst if you combine them together. They are “building blocks” that you as a developer can use to build “your own version” of an integrated Developer Toolchain.
In addition to that they live in a specific AWS account, as mentioned above, which makes the handling of access complicated and requires you to have an IAM user that is allowed to access them.
The user interface and possible integrations are minimal and feel “developer unfriendly”. CodeCommit has the CodeGuru Reviewer integration which is currently not available in CodeCatalyst.
CodeBuild (and with that CodePipeline) is very slow in bringing up new, fresh “build instances” – so starting a new pipeline execution can take minutes which is bad for developer productivity. This is something that CodeCatalyst is addressing with the “lambda” execution environment.
Summary, takeaways and our wishes
As per the messaging, blog posts and announcements from AWS around CodeCatalyst, we believe that the service today aims to offer an opinionated tool for development teams that want to practice “You build it, you run it” – in line with the DevOps mentality. It also means that AWS shows the courage to not only give builders a tool at hand but also “influences” what they build with Blue Prints that include best practices. The vision for CodeCatalyst however could be even more than that: a tool, powered by KI capabilities that empowers builders to efficiently develop and build high quality software by reducing the manual work and efforts through automation.
However, CodeCatalyst is not yet there and it’s going to take some time and effort from the team to reach this.
Wishes for Developer Tooling in General
This post has shown that AWS offers a lot of different possibilities to handle software projects on AWS. We made clear that all of the available tools serve a different purpose and target a different audience. While Amplify focusses on Web or Mobile developer and Application Composer targets Serverless developers, Code Catalyst takes a more generalist approach.
Overall, the “Developer Tools” landscape on AWS needs:
More and better guidance on WHEN to use WHICH service
Better “HOW TOS” instead of hard-to-read documentation or specification
Wishes for CodeCatalyst
Compiling a wish-list for CodeCatalyst can be a big effort as there are still a lot of features that we would like to see. We’ll touch on a few ones here:
General
Single Sign On without Builder ID – Okta/Ping/etc.
Other regions support
Allow “Open Source” projects
Issues / Tracking
Epics
Roadmap / Timeline
Integration with Workflows & Automation
Source
Import projects from Git providers
Automations on Pull Request
CodeGuru
Security Review
Best Practice Review
Support of pre-commit hooks when editing online
Verifications, linting, etc. automated
Workflows
More triggers (e.g. by PR, by schedule, by API)
Conditional Steps
Manual approvals
App Store / Play Store deploy actions
Projen Action
Better integration with AWS services
Import existing CodePipelines
Pipeline as Code – CDKPipelines like option to create workflows from code
What wishes do YOU have for Code Catalyst? What is your “most hated” or “most loved” feature today?
