A real project with CodeCatalyst – Our hackathon gave us a good insight into what works and what doesn’t

As Matt already wrote in his project introduction post we have been using Amazon CodeCatalyst to handle all of our project activities. This gives a very good indication into where CodeCatalyst is already “ready for prime time” and where further adjustments need to be made in order to develop the service into a fully fledge, all-or-nothing, integrated DevOps service.
The one pushing for using CodeCatalyst was myself, as I wanted to try out the new Amazon service that recently went “GA” after being announced at re:invent 2023 in a bigger team and in a real project. It was great fun – and we learned a lot, too!
Let’s look at some details.

How we planned our tasks using “issues”

Our journey with CodeCatalyst started with a – very agile and intense – planning session using the “issues” component of CodeCatalyst. Given the nature of an hackathon, we created roughly the first ten issues in the system. Three of us were “new users” for CodeCatalyst, but they were able to quickly interact with the issues component – by default, this is a simple Kanban board and has a well structured user interface. One thing we directly missed was the ability to copy/paste images into comments within the issues…and to attach files to the issues themselves. Linking issues to one another is also only hardly possible right now.
We used a simple workflow and did a regular planning (twice per week) and created/updated/closed more than 50 issues (of partly very small scope) throught the course of less than four weeks. Overall, the available functionalities were good enough for our project and supported our project.

Collaborating source code – working with Git repositories and pull requests

The four of us are (Community) Builders from the bottom and all we need to be happy is a Git repository that we can push code to 🙂
Of course this is not currect, but CodeCatalyst allows you to host you own, private Git repositories. You can have multiple repositories per project. In our case, we had decided to go with a mono-repo approach and so we strickly worked in a single repository. Working with the source code repository was “just fine” and working as expected. We also decided to merge as early and as often as possible to our “main” branch. The team however struggled in working with setting up Pull Requests and working in the UI when reviewing pull requests.
The most problematic things: UI response time, working with comments on pull requests (e.g. threaded comments) and creating pull requests. The whole UI does not yet feel as “structured” as in other competitors (e.g. Github) – examples: PR title/description proposals when creating a PR, link in terminal/commandline that allows your to create a PR.
Another thing we missed is “git blame” online – not to blame each other but to understand what changed and broke our “hacked” “hackathon” source code 🙂
Cool: the markdown rendering for files named *.md. Missing here: including local images is not possible.

CodeCatalyst usage increased a lot in this project!

Working with Workflows – Github actions for the rescue!

We started off building our workflows with natively available actions in Amazon CodeCatalyst – using the “cdk deploy” action to deploy out infrastructure and the “s3 deploy” action to deploy our front end – but quickly shifted away from that and switched over to using Github Actions within the workflow to allow building our Flutter application as a CDK asset. As our Continuous Integration pieces matured within our npm implementation, there was less reason to go back to the native actions as the CI part was handled through npm.
Our current workflow and CI/CD pipeline:



The main preference here is to allow a continuous deployment to our production environment. This goes through an intial “sandbox” deployment, promotes to a “test” environment and then deploys to “production”. The pipeline is prepared to execute integration tests, but they are not yet implemented – which is the nature of a _hack_athon 🙂
This would definately one of the next things to change – adding real integration tests and also adding some security verifications.

Our workflow is, as I mentioned, pretty basic – here an excerpt of it until the deployment to our sandbox environment:

BuildAndTestCDKApp:
    Identifier: aws/github-actions-runner@v1
    Compute:
      Type: EC2
    Inputs:
      Sources:
        - WorkflowSource
      Variables:
        - Name: AWS_REGION
          Value: us-east-1
        - Name: NODE_ENV
          Value: sandbox
    Outputs:
      AutoDiscoverReports:
        Enabled: true
        ReportNamePrefix: rpt
        IncludePaths:
          - ./coverage/lcov.info
    Configuration:
      Steps:
        - name: Flutter Build Web
          uses: subosito/flutter-action@v2.8.0
          with:
            channel: stable
        - name: Java Install
          uses: actions/setup-java@v2
          with:
            distribution: zulu
            java-version: "11"
        - name: Setup Android SDK
          uses: android-actions/setup-android@v2
        - uses: actions/setup-node@v3
          with:
            node-version: 16
        - run: npm ci
        - run: npm run
        - run: npm run synth
    Environment:
      Connections:
        - Role: CodeCatalystWorkflowDevelopmentRole-AWSCommunityBuilders
          Name: community-builders-sandbox
      Name: speakers_sandbox
  DeploySandboxCDKApp:
    DependsOn:
      - BuildAndTestCDKApp
    Actions:
      DeployBase:
        Identifier: aws/github-actions-runner@v1
        Compute:
          Type: EC2
        Inputs:
          Sources:
            - WorkflowSource
          Variables:
            - Name: AWS_REGION
              Value: us-east-1
            - Name: NODE_ENV
              Value: sandbox
        Configuration:
          Steps:
            - name: Flutter Build Web
              uses: subosito/flutter-action@v2.8.0
              with:
                channel: stable
            - name: Java Install
              uses: actions/setup-java@v2
              with:
                distribution: zulu
                java-version: "11"
            - name: Setup Android SDK
              uses: android-actions/setup-android@v2
            - uses: actions/setup-node@v3
              with:
                node-version: 16
            - run: npm ci
            - run: npm run deploy
        Environment:
          Connections:
            - Role: CodeCatalystWorkflowDevelopmentRole-AWSCommunityBuilders
              Name: community-builders-sandbox
          Name: speakers_sandbox
  IntegrationTestSandbox:
    Identifier: aws/build@v1
    Inputs:
      Sources:
        - WorkflowSource
    Configuration:
      Steps:
        - Run: ls -al
    Compute:
      Type: Lambda
    DependsOn:
      - DeploySandboxCDKApp

As you can see, a lot of the CD (and also the “deployment”) runs within the corresponding task definitions in package.json.

No “Open Source” or “Readonly” view

We would really love to open-source our whole projet, to give you the possibility to get involved and learn from what we build, but unfortunately Amazon CodeCatalyst does currrently not offer an Open Source or a “Readonly” view and forces you to create an account for CodeCatalyst – these are things that make it difficult for us as we cannot link to our sources.
If you are an AWS Community Builder, please reach out to either of us in Slack and we can give you access to the project.

What did we really miss in the workflow component

  • manual approvals before promotion to our “production” environment
  • Notifications from workflows – could have been done through this implementation but we did not take time to implement that
  • Integration of actions deployed through “npm – cdk deploy” with making the changes in infrastructure visible
  • importing existing pipelines or render CDK pipelines to CodeCatalyst workflows The “killer feature” of the workflows, at least as far as we see it, is the possibility to use Github actions as part of the workflow. With only minor adjustmends an existing Github workflow can be transformed to a CodeCatalyst workflow. If either of you know about an existing tool that converts existing Github workflows to CodeCatalyst workflows in an automated way – please let me know.

The hidden gem: Development Environments

One of the “hidden gems” of CodeCatalyst are the Dev Environments – also known as Development Enviroments. Essentially, its a “service in the service” similar to Gitpod where you can host your IDE or the environment that you develop on. In our project we all didn’t use it as the architecture was small and we only had this single project. Also, we were developing mainly on our main machines without switching between machines too often. That’s why we all did not consider really using the DevEnvironments for this project. Last but not least, Flutter is not natively supported on the Dev Environments which would have forced us to prepare a devfile that includes the Flutter dependencies – and thats something that would have taken too much time out of the project.
Still, the De Environments within CodeCatalyst are hidden gems that I believe should get more traction than they are getting until now.

A summary of what we think works and what doesn’t – and our biggest wishes

For a hackathon like this (3-6 weeks working time) CodeCatalyst seems to be a very good choice for a project that starts with a “new” or “fresh” codebase. The tool has most of the minimal requirements implemented to allow simple project management and good integration with AWS.
It also allowed us to quickly get started and to collaborate on our sources.

We really missed workflow notifications as well as manual approvals in the workflows.
As the tool matures I personally hope that more and more AWS internal teams will be using CodeCatalyst to develop, plan and deploy out their internal AWS services and with that the “flow” in the User Interface will definately be improved, too – as today there are some hickups in a real developers workflow.

And, last but not least – we would love to share this project with you, but given CodeCatalyst does not allow to “Open Source” a project, that’s not possible 🙂

Maybe something for the next hackathon?

If you would like to get involved in this project and help us to shape and build our AWS Speakers directory to make it easier for AWS User Group Leaders to find speakers – please reach out to either of us four about collaborating!

Who to reach out to?
DanielleMattJulian or myself.

Views: 1693

Amplify SDK for Flutter – Developer experience and challenges in a hackathon

Background of this post

This blog post is part of a a series of posts that explains details and technical challenges that we (Danielle, Matt, Julian and myself) faced during an hackathon that was focused on the Transformers Huggingface tools – please read Matt’s article for additional information and details.
In this post we are focusing on the experiences we made in the project using the Amplify SDK for Flutter.

Project Setup – architecture dependencies for project set up

Initially our project is targetting to be available to all AWS User Group Leaders on the web from any browser. As we mature the project, at least I am hoping to be able to also publish this application as an Android and iOS application using cross platform capabilities.
Here is a birds eyes view to our UI architecture:


Why did we choose Flutter as UI?

Flutter is a trending cross-platform development toolkit, altough it is mainly sponsored by Google there is a vibrant open source community. I personally have had some exposure to Flutter in the past and I liked the developer experience and the short iteration cycles. Also the AWS Amplify Team has been investing into making their SDK available to developers, so we thought this is a good possibility to try out our use case and implement the Web app in Flutter using the Amplify SDK for Flutter to connect to the backend resources which we were planning to implement using the AWS CDK. With this approach, we also want to draw some attention on the fact that the Amplify SDKs can be used without an Amplify owned backend. This is cool and opens up a lot of possibilities – but also challenges as we will see later. I convinced Danielle, Matt and Julian to also use Flutter for our frontend. They also saw this as a good opportunity to learn a bit of Flutter by themselves.

Amplify SDK for Flutter – Developer experience and challenges

The Amplify SDK for Flutter recently announced the “GA” for Web and Desktop which is a milestone for the team to reach. As we outlined in Matt’s blog post, we are using Typescript in the backend. Amplify Flutter has native support for AppSync/GraphQL – we needed to connect the Flutter App to existing AppSync endpoints.
The AppSync schema however was written manually. No, we needed to use “amplify codegen” to generate the Dart models for the GraphQL types – but we also needed to write a type model in Typescript to be able to work with the same objects on the backend.
This turned out to be more difficult than expected: the [amplify codegen](https://docs.amplify.aws/cli/graphql/client-code-generation/#shared-schema-modified-elsewhere-eg-console-or-team-workflows) functionality is available for Flutter, too, but it was difficult to get this to work.
We ended up creating a new Flutter application using amplify init and then copy/pasted our schema(s) into the expected location. Then we needed to manually copy the generated models into our project.
Oh, I neqarly forgot: When using amplify codegen you need to ensure that your schema is anotated correctly, types e.g. need to have an @model annotation – but if you have this annotation in the schema when trying to deploy to AppSync, that deployment failed…so we needed to also manually adjust the schema before we were able to execute amplify codegen.

Using the Amplify SDK for Flutter

In our use case, all of the backend infrastructure is created using the AWS CDK. We are not using Amplify to create backend resources – and this use case has – until recently – not really been promoted by the Amplify team. Thanks to one of my last blog posts around the same topic, this new documentation page has been added which simplifies the setup of your amplifyconfiguration.dart – but we were missing the same documentation for the “Authentication” library. One again, we workarounded this problem by using a temporary Amplify Flutter project. That allowed us to copy/paste the configuration and adjust it to our needs.

Environment aware connections

In our current setup, we have at least three environments to test and promote our application. In order to be able to execute and test the Flutter application on all environments without code changes, we needed to make the maplifyconfiguration.dart environment aware:

class EnvironmentConfig {
    static const WEB_URL = String.fromEnvironment('WEB_URL');
    static const API_URL = String.fromEnvironment('API_URL');
    static const CLIENT_ID = String.fromEnvironment('CLIENT_ID');
    static const POOL_ID = String.fromEnvironment('POOL_ID');
}

These environment variables are then used in our configuration object:

const amplifyconfig = """{
"UserAgent": "aws-amplify-cli/2.0",
    "Version": "1.0",
    "api": {
        "plugins": {
            "awsAPIPlugin": {
                "frontend": {
                    "endpointType": "GraphQL",
                    "endpoint": "${EnvironmentConfig.API_URL}",
                    "region": "us-east-1",
                    "authorizationType": "AMAZON_COGNITO_USER_POOLS"
                }
            }
        }
    },
    "auth": {
        "plugins": {
            "awsCognitoAuthPlugin": {
                "UserAgent": "aws-amplify-cli/0.1.0",
                "Version": "0.1.0",
                "IdentityManager": {
                    "Default": {}
                },
                "CognitoUserPool": {
                    "Default": {
                        "PoolId": "${EnvironmentConfig.POOL_ID}",
                        "AppClientId": "${EnvironmentConfig.CLIENT_ID}",
                        "Region": "us-east-1"
                    }
                },
                "Auth": {
                    "Default": {
                        "authenticationFlowType": "USER_PASSWORD_AUTH",
                        "socialProviders": [],
                        "usernameAttributes": [
                            "EMAIL"
                        ],
                        "signupAttributes": [
                            "BIRTHDATE",
                            "EMAIL",
                            "FAMILY_NAME",
                            "NAME",
                            "NICKNAME",
                            "PREFERRED_USERNAME",
                            "WEBSITE",
                            "ZONEINFO"
                        ],
                        "passwordProtectionSettings": {
                            "passwordPolicyMinLength": 8,
                            "passwordPolicyCharacters": []
                        },
                        "mfaConfiguration": "OFF",
                        "mfaTypes": [
                            "SMS"
                        ],
                        "verificationMechanisms": [
                            "EMAIL"
                        ]
                    }
                }
            }
        }
    }
}""";

Within our CI/CD workflow we then pass the correct values for the variables and during the flutter build web they are backed into the application.

After solving these challenges, using the Amplify Flutter library worked without noteworthy problems or hickups.

Using the Amplify Flutter Library to authenticate a user with Cognito

The documentation for Amplify Flutter is really good and we decided to also use the Authenticator Widget – at the end, this project was born through a hackathon – and we did not have much time to implement the authentication flow ourselves.

In our main.dart we needed to include the Amplify configuration:

Future<void> _configureAmplify() async {
  final api = AmplifyAPI(modelProvider: ModelProvider.instance);
  await Amplify.addPlugin(api);

  // Add any Amplify plugins you want to use
  final authPlugin = AmplifyAuthCognito();
  await Amplify.addPlugin(authPlugin);

  try {
    await Amplify.configure(amplifyconfig);
  } on AmplifyAlreadyConfiguredException {
    safePrint(
        'Tried to reconfigure Amplify; this can occur when your app restarts on Android.');
  }
}

and then we only needed to adjust the build method to distinguish betwen AuthenticatedViews and “normal” views:

Widget build(BuildContext context) {


   final GoRouter router = GoRouter(
      routes: <RouteBase>[
        GoRoute(
          path: '/',
          builder: (BuildContext context, GoRouterState state) {
            return HomeScreen();
          },
        ),
        GoRoute(
          path: '/profile',
          builder: (BuildContext context, GoRouterState state) {
            return AuthenticatedView(
                child: ProfileScreen(),
            );
          },
        ),
        GoRoute(
          path: '/imprint',
          builder: (BuildContext context, GoRouterState state) {
            return ImprintScreen();
          },
        ),
        GoRoute(
          path: '/privacy',
          builder: (BuildContext context, GoRouterState state) {
            return PrivacyScreen();
          },

        ),
        GoRoute(
            path: '/in',
            builder: (BuildContext context, GoRouterState state) {
              return AuthenticatedView(
                  child: LoggedInHomepage(title: 'AWS Speakers Directory'));
            },
        ),
      ],
    );

    return Authenticator(
      initialStep: AuthenticatorStep.signIn,
      signUpForm: SignUpForm.custom(
        fields: [
          SignUpFormField.username(),
          SignUpFormField.password(),
          SignUpFormField.passwordConfirmation(),
          SignUpFormField.email(),
          SignUpFormField.custom(
              title: "First name",
              attributeKey: CognitoUserAttributeKey.givenName),
          SignUpFormField.custom(
              title: "Last name",
              attributeKey: CognitoUserAttributeKey.familyName),
          SignUpFormField.custom(
              title: "City",
              attributeKey: CognitoUserAttributeKey.custom("city")),
          SignUpFormField.custom(
              title: "Country",
              attributeKey: CognitoUserAttributeKey.custom("country")),
          SignUpFormField.birthdate(),
          SignUpFormField.gender(),
        ],
      ),
      child: MaterialApp.router(
        title: 'AWS Speakers Directory',
        routerConfig: router,
      ),
    );
  }

We would have loved to open source our code, but unfortunately that option is currently not available in Amazon CodeCatalst.
If you’re an experienced Flutter developer and don’t like the code you see above – that’s fine, the four of us have been learning Flutter iwth this project – approach us and tell us how to do this better! 😉

Using the Amplify Flutter Library to execute AppSync / GraphQL queries

Accessing AppSync was also really easy by following the Amplify Flutter documentation. Here is our code for retrieving events from the backend:

Future<List<EventRow?>> _listEvents() async {
    try {
      String graphQLDocument = '''query GetEvents {
      listEvents {
        pk
        sk
        title
        description
        tags
        length
      }
    }''';

      var operation = Amplify.API
          .query(request: GraphQLRequest<String>(document: graphQLDocument));
      List<EventRow> events = [];
      var response = await operation.response;
      var data = response.data;
      if (data != null) {
        Map<String, dynamic> userMap = jsonDecode(data);
        print('Query result: ' + data.toString());
        List<dynamic> matches =
            userMap["listEvents"] != null ? userMap["listEvents"] : [];
        matches.forEach((element) {
          if (element != null) {
            if (element["id"] == null) {
              element["id"] = "rnd-id";
            }
            var event = Event.fromJson(element);
            events.add(EventRow(event, context));
          }
        });
        return events;
      }
    } on ApiException catch (e) {
      print('Query failed: $e');
    }

    return <EventRow?>[];
  }

This uses the previously generated models. All of the queries are authenticated using the Cognito Authentication – and this is the identity that we can also use in the backend to authorize access.
The Amplify documentation is pretty good, hence I will not be adding more details into this post.
Please ask if you have any specific questions.

Wishes for the Amplify SDK for Flutter

We already mentioned most of them: better amplify codegen support (including the option to generate both Flutter and Typescript models at the same time), better documentation or support for setting up the Amplify configuration completely without an Amplify backend.
Another thing we did not talk about here but Julian mentions in his blog post are the AppSync [merged APIs] which are currently not supported – thus we needed to execute amplify codegen for each of our microservices schemas and then for the merged APi – and then manually needed to bring the models into a usable structure (e.g. copy/pasting from the different ModelProviders).

What’s next for our project

So, what’s next? After the hackathon we are thinking about making this a “kind of” OpenSource, collaborative project. Here we are looking for contributors – please contact us if you are interested.
Besides that, Matt covers a godd bunch of roadmap items already – and we definately need someone with more Flutter experience to review our UI code to e.g. introduce the BloC pattern for cleaner programming, adding some styling and to expand existing functionalities.
And we still need to play the “cross-platform” card – we need a build step to generate iOS and Android versions of our application and need to make sure that the apps land in the Appstore(s)!
Please reach out to us if you would like to get involved!

Who to reach out to?
DanielleMattJulian or myself.

Views: 446