Home > On-Demand Archives > Workshops >
Getting Started with Embedded DevOps using Gitlab CI/CD Pipelines
Jacob Beningo - Watch Now - EOC 2024 - Duration: 01:47:27
Continuous Integration and Continuous Delivery (CI/CD) have become critical tools to IoT edge device developers. In this workshop, participants will delve into the fundamentals of Embedded DevOps by designing and implementing their own CI/CD pipeline using Gitlab.
Attendees will gain practical experience in configuring build systems, designing a CI/CD pipeline, and implementing it. (At least as much as can be done in a few hours). We’ll explore how to containerize your build environment in Docker, so that you can easily integrate it into an embedded CI/CD pipeline. You’ll also learn how to use Visual Studio Code to seamlessly integrate your build processes within a single environment.
Attendees will walk away with a basic, but functional CI pipeline that they can easily scale to meet their needs.
Key topics covered in this workshop include:
- The role of DevOps in Edge and embedded system development
- CI/CD pipeline design for embedded systems
- Containerizing your build system in Docker
- Set up and deployment of CI/CD solutions
- Best practices and steps to go further
So finishing working through this workshop now that I have a bit of time. In part 2, where we set up the CI/CD pipeline, noticed some minor bugs:
- The
gitlab-ci.txt
file has the lineimage: beningo/embedded-dev:latest
in the default section. Thebeningo
should bebeningojw
. Otherwise this causes a build failure when looking for the image on gitlab. - The
unit-test-job
stage in the.gitlab-ci.txt
file callsmake tests
instead ofmake unit_tests
. - There is already a completed
.gitlab-ci-yml
file present in the/00-Starters/LAB_2/controller/
, not sure if that got left over from the solutions folder.
Not sure if you're going to use these files again for another workshop or not, but wanted to let you know.
Enjoyed the workshop and it's great to see how all this works. Will be hoping to work more with CI/CD in the future for my projects! (seems like my markdown code highlighting might not display properly on the website comments)
Thanks for letting me know. I’ll double check if these changes have already been made in my full course.
I’m glad to hear that you were able to take away several actions to apply in your projects!
It's been reported in an Ars Technica article this week that GitLab has a critical security vulnerability.
https://arstechnica.com/security/2024/05/0-click-gitlab-hijacking-flaw-under-active-exploit-with-thousands-still-unpatched/
Thanks for letting us know!
11:07:47 From Stephane to Everyone: Do the slides look clear to everyone? They seem to be a bit blurry for me. 11:08:09 From Ozcan Yurt to Everyone: Reacted to "Do the slides look c..." with 👍 11:08:21 From Scott H to Everyone: Replying to "Do the slides look c..." yeah quality is a bit pixelated 11:08:23 From Brian Schmalz to Everyone: They are blurry for me as well 11:08:32 From Ian Rasmussen to Everyone: Replying to "Do the slides look c..." looks like a low bit-rate connection. 11:08:35 From Caio J. B. V. Guimaraes to Everyone: Quality is bad right now 11:08:44 From Ozcan Yurt to Stephane(Direct Message): Blurry, too. 11:08:51 From Stephane to Ozcan Yurt(Direct Message): Here we go! 11:08:53 From Caio J. B. V. Guimaraes to Everyone: now it is way better 11:08:55 From Ian Rasmussen to Everyone: yep 11:08:56 From Ozcan Yurt to Everyone: Removed a 👍 reaction from "Do the slides look c..." 11:09:13 From Scott H to Everyone: Reacted to "now it is way better" with 👍 11:09:15 From Singhal, Rishi to Everyone: it did! 11:21:10 From Stephane to Jacob Beningo(Direct Message): FYI: Right now, the part of the Zoom where we see you contains also the slides (so we see the slides twice, once through your screen sharing, and twice through your camera feed) 11:24:21 From Caio J. B. V. Guimaraes to Everyone: You could use podman... 11:24:28 From Jeff Cassidy to Everyone: Reacted to "You could use podman..." with 👍 11:25:06 From Andrew MacIsaac to Everyone: Reacted to "You could use podm..." with 👍 11:25:30 From Robert Hancock to Everyone: Reacted to "You could use podman..." with 👍 11:26:22 From Shilpa to Everyone: Reacted to "You could use podman..." with 👍 11:30:03 From tyhw to Everyone: FYI Docker does require a reboot (just found that out 5min ago lol) 11:30:21 From Shilpa to Everyone: Reacted to "FYI Docker does requ..." with 👍 11:31:51 From Nathan to Everyone: Docker Desktop may require commercial license depending on your company (https://www.docker.com/products/docker-desktop/) 11:33:21 From Tom to Everyone: use caution when using the package manager to install critical tools as you can get unexpected version changes between image builds. 11:34:49 From Scott H to Everyone: Reacted to "use caution when usi..." with 👍 11:36:21 From Jacob Beningo to Stephane(Direct Message): RUN cd /home/dev && \ curl -LO https://developer.arm.com/-/media/Files/downloads/gnu/13.2.rel1/binrel/arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz && \ tar xf arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz && \ rm arm-gnu-toolchain-13.2.rel1-x86_64-arm-none-eabi.tar.xz 11:37:36 From Scott H to Everyone: to me the main fragility of docker is just that - bitrot. Aside from running your own file server is there a nice way to combat that? 11:40:00 From Singhal, Rishi to Everyone: sorry I might have missed this since I joined 5 mins late-where do we get these files? 11:40:40 From Shilpa to Everyone: Replying to "sorry I might have m..." same 11:40:46 From Eric to Everyone: @@Singhal, Rishi https://embeddedonlineconference.s3.amazonaws.com/eoc/files/EOC_CICD_Workshop_Materials.zip 11:40:56 From Singhal, Rishi to Everyone: thanks! 11:41:30 From Scott H to Everyone: I guess my concern is when I go back to revisit an old project ~5 years later, will go with the file server option. Cheers! 11:41:46 From John W to Everyone: you could also host forks of images in your own git repo for example, in the event that you didn't want to host your own file server, e.g. on github. Unless I am missing something in that question/response about bit rot 11:41:52 From Caio J. B. V. Guimaraes to Everyone: Sometimes auditions happens...5 to 10 years later 11:42:35 From Caio J. B. V. Guimaraes to Everyone: *audit 11:43:12 From Scott H to Everyone: Reacted to "you could also host ..." with 👍 11:43:31 From Andrew MacIsaac to Everyone: If you make the container image itself an artifact, then you can build and store it, rather than rebuilding it on use. If you make your CI/CD and your developers pull the image from a (potentially local) container repository, then you can have all the dependencies bundled together. 11:43:32 From Nathan to Everyone: Be careful about licenses for some of these (like vendor specific stuff): you don't always have the right to make them available yourself 11:44:20 From John W to Everyone: Reacted to "Be careful about lic..." with 👍 11:44:43 From Tom to Everyone: Reacted to "use caution when u..." with 👍 11:44:44 From Tom to Everyone: Removed a 👍 from "use caution when u..." 11:47:58 From tyhw to Everyone: I'm a bit confused. What is it we are actually trying to do here? 11:49:27 From Ian Rasmussen to Everyone: Replying to "I'm a bit confused. ..." If you have downloaded the folder, in the Labs folder there's a PDF called EOC_CICD_Workshop_Exercises.pdf that contains the "directions" 11:49:49 From tyhw to Everyone: Replying to "I'm a bit confused. ..." Oh! Thanks 11:55:00 From Ian Rasmussen to Everyone: When you say "The more RUN commands you use the more layers will exist in your image." does that mean any invocation or RUN adds a layer, or any individual command used in RUN (like RUN apt-get update) is a new layer? 11:55:29 From Ian Rasmussen to Everyone: the spaghetti is calling to me with one long RUN command 11:58:40 From Ian Rasmussen to Everyone: yep, thanks 12:01:22 From Ian Rasmussen to Everyone: So if I understand docker correctly, we create the Dockerfile to assemble the image, and then the image is distributed to developers, not the Dockerfile, ensuring that everyone is on the same version of code? I ask since I see the apt-get update commands in the RUN commands. 12:04:41 From Ian Rasmussen to Everyone: thank you 12:05:14 From Jacob Beningo to Stephane(Direct Message): Built Image 12:05:20 From Jacob Beningo to Stephane(Direct Message): Building Image 12:05:27 From Jacob Beningo to Stephane(Direct Message): Still Working 12:05:29 From Jacob Beningo to Stephane(Direct Message): Done 12:05:53 From Jacob Beningo to Everyone: Built Image 12:05:55 From Jacob Beningo to Everyone: Building Image 12:05:57 From Jacob Beningo to Everyone: Still WOrking 12:05:58 From Jacob Beningo to Everyone: Done 12:06:02 From Scott H to Everyone: Reacted to "Still WOrking" with 👍 12:06:04 From Singhal, Rishi to Everyone: Reacted to "Still WOrking" with 👍 12:06:06 From tyhw to Everyone: Reacted to "Still WOrking" with 👍 12:06:13 From Lyden Smith to Everyone: Reacted to "Building Image" with 👍 12:06:14 From Ryan to Everyone: Reacted to "Building Image" with 👍 12:06:23 From Ian Rasmussen to Everyone: Reacted to "Building Image" with 👍 12:06:36 From Tom to Everyone: Reacted to "Done" with 👍 12:06:43 From Ali to Everyone: Reacted to "Still WOrking" with 👍 12:06:54 From Ali to Everyone: Reacted to "Building Image" with 👍 12:07:22 From tyhw to Everyone: Removed a 👍 reaction from "Still WOrking" 12:07:24 From tyhw to Everyone: Reacted to "Building Image" with 👍 12:07:25 From Nick Tillich to Everyone: Should step 15 say ''YourLastName/cicd-dev:latest bash'' instead of cpp-dev? 12:07:34 From Nick Tillich to Everyone: Reacted to "Done" with 👍 12:09:08 From Lyden Smith to Everyone: Reacted to "Built Image" with 👍 12:09:09 From Lyden Smith to Everyone: Removed a 👍 reaction from "Building Image" 12:09:28 From Lyden Smith to Everyone: Reacted to "Should step 15 say '..." with 👍 12:10:15 From Lyden Smith to Everyone: Removed a 👍 reaction from "Built Image" 12:10:17 From Lyden Smith to Everyone: Reacted to "Done" with 👍 12:13:31 From KK to Everyone: Reacted to "Building Image" with 👍 12:15:03 From Scott H to Everyone: (Am getting an oath token error when I try docker build, any hints on the bet path?) 12:22:45 From Lyden Smith to Everyone: Does the screenshare look blurry to anyone else? 12:23:08 From Brian Schmalz to Everyone: It looks clear to me 12:23:33 From Lyden Smith to Everyone: Oh I had the wrong view, thanks! 12:29:06 From Singhal, Rishi to Everyone: Can someone resend the ARM toolchain link for gcc-arm-none-eabi? I had to restart my computer to get docker installed and lost that from the chat history. The one on the guide isn't working 12:32:31 From Ian Rasmussen to Everyone: Replying to "Can someone resend t..." https://developer.arm.com/-/media/Files/downloads/gnu-rm/10-2020q4/gcc-arm-none-eabi-10-2020-q4-major-x86_64-linux.tar.bz2 This style of one? (This one is from the guide, works for me). Or referring to a different link? 12:33:05 From Singhal, Rishi to Everyone: Replying to "Can someone resend t..." yes. Ok good to know it works for you, it's probably a copy paste error or something 12:33:42 From Nick Tillich to Everyone: Replying to "Can someone resend t..." if you're copy-pasting from PDF, careful as it's turning --version into -version 12:33:59 From Scott H to Everyone: Replying to "(Am getting an oath ..." This was resolved by using the docker gui instead of the command line 12:35:43 From Ian Rasmussen to Everyone: There's a Linux CLI version of Docker which I believe avoids the licensing of the Windows Desktop version (or at least that's the understanding I've been operating under for the last year or so 😅). Makes it easy to use with WSL, but I remember it being a little bit of a pain to get installed. 12:38:26 From Austin K (Indesign) to Everyone: Anyone else get this error after installing Docker and trying to run Docker desktop? 12:39:54 From Scott H to Everyone: Removed a 👍 reaction from "Still WOrking" 12:39:57 From Scott H to Everyone: Reacted to "Done" with 👍 12:42:01 From Ian Rasmussen to Everyone: Replying to "Anyone else get this..." Do you have WSL 1 or WSL 2? Pretty sure it only works with WSL2 (not sure if that would generate this error specifically, just know that there will be issues if it's WSL 1) 12:42:13 From Austin K (Indesign) to Everyone: Here is the full error message: provisioning docker WSL distros: ensuring main distro is deployed: checking if main distro is up to date: checking main distro bootstrap version: getting main distro bootstrap version: exit code: 4294967295: running WSL command wsl.exe C:\Windows\System32\wsl.exe -d docker-desktop -u root -e wsl-bootstrap version: Failed to attach disk '\AppData\Local\Docker\wsl\distro\ext4.vhdx' to WSL2: The system cannot find the path specified. Error code: Wsl/Service/CreateInstance/MountVhd/HCS/ERROR_PATH_NOT_FOUND : exit status 0xffffffff checking if isocache exists: CreateFile \\wsl$\docker-desktop-data\isocache\: The network name cannot be found. 12:42:52 From Austin K (Indesign) to Everyone: I have WSL 2. I think my Docker install didn't work, as I'm missing "AppData\Local\Docker\" folder, so it can't find that path. 12:43:06 From Scott H to Everyone: Gotta go, thanks for the workshop, will continue experimenting in my own time. Looks easier than I'd assumed which is nice! 12:44:46 From Jacob Beningo to Everyone: Ensure that Docker Desktop is set to use WSL 2 as the backend. In Docker Desktop, go to the "Settings" menu and make sure WSL 2 is selected. 12:45:49 From Austin K (Indesign) to Everyone: Okay, WSL 2 is selected. I will try to reinstall Docker. Thanks for looking. 12:47:52 From Ian Rasmussen to Everyone: I've had no issues running it on WSL2 for the past year+ (was just using it as a build environment provided by another company, so no experience building anything out). Setup was a bit rough, and then jumping through some hoops to get USB uploading working. But has been great since. 12:49:17 From Michael to Everyone: My host machine is Ubuntu 22.04 LTS and i'm installing docker and docker desktop for the first time. I followed the instructions in docs.docker.com for installation and have disabled docker engine (per the suggestion on the install docs page). When I laundh docker desktop it says starting the docker engine and that never finishes (15+ mins, two relaunches). I haven't rebooted as I don't want to drop this call at this moment. Any suggestions for things to look into for why docker desktop doesn't appear to start? 12:50:16 From Ian Rasmussen to Everyone: Replying to "My host machine is U..." Someone at the beginning said that desktop needs a reboot after install, but they didn't specify which OS. So that might just be your issue. 12:50:42 From Michael to Everyone: Yea, they mentioned WSL so I was thinking that was windows but perhaps it is linux too 12:51:07 From Michael to Everyone: if no obvious gotchas anyone knows I'm sure I can sort it out later/offline 12:54:08 From Austin K (Indesign) to Everyone: Do you run build/analysis tools locally before committing to prevent pipeline failures or let the pipeline catch all issues? 12:54:47 From Michael to Everyone: What are the best practices (or the practices you find the most useful) when doing CI/CD development with regards to local testing before committing as well as git commit message standards? Our team is looking to shift to a CI/CD pipeline. We like all this opens up but with hardware in the loop development/testing, would it make sense to use the CI/CD as a validation + release phase vs debug development? A big plus is the centralized build system and tracking this provides but do you feel some development would be better aided running locally first to determine the direction of what to commit and run through the pipeline? 12:57:25 From Jacob Quant to Everyone: For commit messages, we love https://www.conventionalcommits.org/en/v1.0.0/ 12:57:31 From Singhal, Rishi to Everyone: looks like curl failed to verify the ssl certificate for the arm compiler and couldn't establish a secure connection to it- did anyone run into that? 12:58:00 From Brian Schmalz to Everyone: Replying to "What are the best pr..." The CI/CD product we used for a recent project allowed us to do EVERYTHING locally that was also done in the code. All of us developers could easily check that things worked before we did a commit/push. It was really sweet. We used EmebOps and it worked very very well for us. 12:58:05 From Andrew MacIsaac to Everyone: Reacted to "For commit message..." with 👍 12:58:17 From Brian Schmalz to Everyone: Replying to "What are the best pr..." Sorry, not 'code' , 'cloud' 13:01:09 From Michael to Everyone: That's great, thank you 13:02:41 From Ian Rasmussen to Everyone: Replying to "What are the best pr..." EmebOps doesn't come up with any results for me, spelling error? 13:02:48 From Eric to Everyone: With the xz-utils was a big security issue ... 13:04:04 From Lyden Smith to Everyone: Thank you Jacob! 13:04:05 From Eric to Everyone: Thank you for this Long session! 13:04:12 From Jacob Quant to Everyone: 👏 13:04:12 From Ian Rasmussen to Everyone: Replying to "What are the best pr..." EmbedOps comes up with something, so likely that 13:04:14 From Andrew MacIsaac to Everyone: Thank you! 13:04:16 From Brian Schmalz to Everyone: Replying to "What are the best pr..." Sorry! Yes : https://dojofive.com/embedops/ 13:04:21 From Nick Tillich to Everyone: Thank you! 13:04:24 From Ian Rasmussen to Everyone: Reacted to "Sorry! Yes : https:/..." with 👍 13:04:27 From Ian Rasmussen to Everyone: Replying to "What are the best pr..." @Brian Schmalz thanks 13:04:35 From Brian Schmalz to Everyone: Reacted to "@Brian Schmalz thank..." with 👍 13:04:38 From Brandon to Everyone: Thank you 13:05:32 From Ryan to Everyone: Thank You!
I like the point about Agile and DevOps - I think of DevOps as the full realization of Agile, or the endgame of it.