Summary
One of the limitations of CodePipeline currently is that there is not a good way to use common components across multiple pipelines. You may want to do this so you do not have to duplicate buildspecs or other files to be used in different repositories. In this post, I will show a strategy for reusing artifacts across multiple pipelines.Repository Build Scripts
Modern “best of breed” CI servers allow you to commit build configuration to your repository. This is nice because your pipeline gets all the features of your SCM system like versioning and change tracking. But what happens when you would like to create “standard” pipelines for developers? It may not be possible for all repositories, but surely we don’t want to create each “docker image push” or “node js library” ad hoc. It would be much easier to publish CloudFormation templates or provide some other method to easily create pipelines for these types of artifacts.
Solution
The following diagram displays a solution to this problem. By adding an intermediary step that will take the source artifact and add files to it, we can reuse buildspec files. These files should be zipped up and stored in an S3 bucket.
Then, the CodeBuild project will be passed the output artifact of the lambda stage instead of the original source artifact. The build stage for modifying the artifact looks like this:
- Name: Build
Actions:
- Name: ModifySource
ActionTypeId:
Category: Invoke
Owner: AWS
Provider: Lambda
Version: 1
Configuration:
FunctionName: ModifySourceArtifactLambda
UserParameters: '<bucketname>/<myfile>.zip'
InputArtifacts:
- Name: Source
OutputArtifacts:
- Name: ModifiedSource
RunOrder: 1
The ModifySourceArtifactLambda
downloads a zip file and combines it with the InputArtifact
and creates a new artifact. This artifact is then referenced by the CodeBuild project.
- Name: DockerBuildPush
ActionTypeId:
Category: Build
Owner: AWS
Provider: CodeBuild
Version: 1
Configuration:
ProjectName: !Ref 'BuildProject'
InputArtifacts:
- Name: ModifiedSource
RunOrder: 2
The CodeBuild project should be configured to use the path to the buildspec as it is present in the zipped bundle.
Summary
The lambda source and instructions for installation are available on github.
The ModifySourceArtifactLambda
was written with the ability to use it for more than just the
buildspec use case. You can imagine it being useful sharing other resources across projects. Perhaps
you could store a linting or test reporting configuration file to be used by the build.
If you find this helpful, give me a shout and let me know how you are using it!