Home > On-Demand Archives > Keynote Presentations >
Flight Software Design for Lunar Exploration: A Case Study
Jacob Beningo - Watch Now - EOC 2023 - Duration: 01:25:36
A typical satellite flight computer will often directly control its propulsion system. Direct control, while convenient, adds hardware and software complexity to a flight computer system that is often already overburdened with duties. In the CAPSTONE mission to the moon, the Stellar Exploration Inc. propulsion module was used to uniquely decouple propulsive maneuvers from the flight computer to a propulsion controller.
This session will explore the techniques and tools employed to create robust flight software for Capstone's propulsion controller. Attendees will gain insight into modern development processes, tools, and techniques through real-world examples and learn how software architecture was broken down and implemented as a quasi-microservices architecture. Additionally, we will showcase how RTOS task code was modularized and how configuration tools were utilized to manage RTOS tasks automatically.
Topics covered in this talk will include:
- Software architecture best practices
- How to write mission configurable flight software
- Modern development software processes such as unit testing, DevOps, etc
- Flight software quality analysis
- Examples from CAPSTONE, an Artemis mission to Lunar orbit
Thanks for the question.
How long memory / or a microcontroller will last will depend on a lot of factors. The radiation environment, the technology used, etc. Usually for a mission calculations are performed over the desired orbit that provide a statistical probability of how often SEU's, etc will occur.
Usually systems will employ memory scrubbing and checking to detect bit flips and other events that are caused by radiation. They are then corrected or may trigger a power cycle of the system.
Thanks for your answer.
I've asked that question because your lecture reminded me of a story once told by an ex-colleague. One day, he or one of his friends were involved in the development of the satellite, which was controlled by the numerous 8-bit microcontrollers (AVR or PIC architecture). I don't remember exactly how many of them there were. And unfortunately, I cannot ask him again, as he moved to the USA long ago. But definitely below 10. Probably that was an educational satellite built in one of the ex-USSR countries.
But I definitely remember that he has emphasized that numerous controllers were put there just for redundancy, so the satellite could survive for a while: if one dies, then there is still a few in reserve.
I was young and didn't believe him at the time. :-) But it seems like a common practice for certain applications.
Redundancy is definitely a technique that is often used in space systems. Education and smaller satellites might not make use of it. Watchdogs are critical and then redundancy for when there is a hardware failure. However, that adds complexity and cost so there are trade-offs that are made depending on what the end goal and life-time of the mission are.
Jacob, your presentation was fascinating. I can identify with your excitement of being a part of the project. In the 1980's, as a cooperative education student working at the General Dynamics facility in San Diego (Convair Division), I worked on the T- software to replace stript chart recorders for pre-launch monitoring. GD was working on adapting the Centaur to fly in the space shuttle bay. That never happened due to the dangers involved with the risk of an explosion of a fully fueled rocket in the bay. I was given a tour of the factory where the Atlas and Centaur were being built. The technician showing me around had me climb up a platform where I found myself next to the Atlas rocket farings. The technician grabbed and pulled me away before the rocket engines underwent gimble testing. It was a setup to impress me with the power and speed at which the engines can gimble. The technician told me, "If you had been standing where you were when the engines gimbaled, you would have been knocked off the platform" (about 15 feet above ground level.) It was an experience I never forgot. I was invited to go to an launch that summer, but decided to return to summer school.
Awesome! Thanks for sharing! Super cool!
One of the top 3 presentations so far at EOC 2023, straight from one of the co-organisers and renown embedded systems luminary. The topic - development of flight software for small satellites is top notch, even for seasoned embedded software development professionals. Concise, detailed presentation even without protected proprietary and confidential details. Without adding more 'noise' by my side - bow down mr. Benigo!
Thanks for the positive feedback. I'm glad to hear that you enjoyed the presentation! Enjoy the rest of the conference!
Following up on a question I put during the Q&A after the keynote:
since the purpose of the software architecture includes presenting the "principles guiding its design and evolution", how does the presented architecture guide the evolution of the Capstone Propulsion Controller ? Could you elaborate imagining new features added to the existing controller and how the architecture would guide us in including these new features ?
ah yes. So, the presented materials probably did not demonstrate this as well as the formalized architecture. There were a key pieces in the architecture that will help it evolve and will guide how to features will be added. (Since launch, the system has actually been scaled dramatically and is expected to be flown in nearly another half-a-dozen missions).
1) You'll remember I mentioned some principles that we used like "there is no hardware", "data dictates design", etc. These principles were used to guide the architecture to use abstractions that allow us minimize application dependencies on the hardware. This allowed us to scale and also simulate the application.
2) You probably can't tell from the diagrams presented, but we leveraged configuration tables and generated code. These allowed for the hardware, valves, pumps, sensors to be easily scaled and exchanged.
3) The architecture leveraged a micro-service like behavior using RTOS threads. New features can be added to the data flow to see what data it needs to consume, what the outputs are, and that will help to guide how that new feature will be implemented. There were communication structures that could be leveraged to broadcast data, which allows new tasks that contain features to be added. This allowed the application to scale without breaking existing infrastructure.
I hope that helps.
That was a great talk and covered lots of material from your book. I was wondering how much of your time it took to optimise your process and tools, vs day-day tasks of coding and testing in order to generate value? Did you have lots of regulatory requirements to fullfil as well, such as DO-178C?
Thanks for the question.
Yes, so a lot of the process and ideas that I use on a day-to-day basis to develop products are represented in the book. To some degree, writing books, articles, etc help me to think through some of these ideas.
In general, I don't consider my processes and tools to ever be efficient or complete. They are constantly evolving, scaling and adapting. I probably spend at least half a day every week thinking about how they working, where they can be improved, what should be changed, etc.
For this project, there were no regulatory requirements for the software. I believe there were some regulations related to storage of the fuel, etc. We did try to follow as robust of a process and code as we believed was necessary to successfully fulfill the mission.
15:02:26 From RF to Everyone: Excellent Presentation let alone excellent product delivery, thank you. 15:03:07 From bblatz to Everyone: Excellent Presentation 15:03:09 From Aaron Olowin to Everyone: Thanks Jacob 15:03:49 From Jacob Beningo to Everyone: https://www.spacedaily.com/reports/CAPSTONE_Demonstrates_Crosslink_Capability_with_Lunar_Reconnaissance_Orbiter_999.html 15:04:26 From Linari to Everyone: How did you set up your target code to run on your computer system? 15:05:03 From Rishav Vatsa to Everyone: what IPC protocols are generally used to communicate between sensors,actuators and MCU 15:05:04 From Douglas Renaux to Everyone: Hi Jacob thanks to the excellent talk Could you go back to slide with the software architecture and explain how it would guide the evolution of the solution ? 15:05:28 From apr to Everyone: Impressive project, thanks for sharing!What was your role resp. parts in this design and development? 15:06:04 From Rishav Vatsa to Everyone: Replying to "what IPC protocols a…" for space communication in specific 15:06:33 From Mohammed Eshaq to Everyone: How is your experience with Embedded File System (such as FatFs) in space applications? Is it recommended? Some might argue that File Systems are fragile and cause FSWs to crash. 15:06:39 From Ribero.Alex to Everyone: Hi. Could you give us an example of some technical difficulty (in firmware) that you ran into during the development, and how you resolve the issue? Thanks! 15:07:11 From George Addison to Everyone: Thank you for the really insightful talk! I was wondering how the details of the communications interface between the thruster controller and flight computer were hammered out? Did you produce an ICD that the flight computer team designed to? Or vice versa? Or somewhere in between? 15:09:15 From David to Everyone: Say you are working at a company (startup of course….) that is bad at everything in the "embedded software triad". Where would you start focus on improving? 15:09:20 From Thomas Schaertel to Everyone: Were you also involved with hardware development? What about redundancy in soft- and hardware? Did you also use radiation safe silicon? Can you also tell something about the costs of just the propulsion system? 15:13:26 From RF to Everyone: You did comment briefly concerning target hardware emulation during the presentation, presumably because the actual final hardware target was not yet available at the time. What are your views ultimately on HIL / virtual modelling tools. 15:16:13 From apr to Everyone: Can you share insights on why your changed from FreeRTOS to whatever-RTOS? 15:22:21 From Aaron Olowin to Everyone: Were you able to end up with a RTOS-agnostic data-sharing scheme between tasks and modules? 15:27:21 From Ulises V to Everyone: thanks 15:27:34 From René Andrés Ayoroa to Everyone: thank you Jacob 15:27:36 From Vishwa Perera to Everyone: Thank you
Thank you for sharing your knowledge! Maybe it's quite a late question, but still.
People often talk about space radiation and its impact on electronics. Especially "deadly" is considered the one the spacecraft meets beyond the LEO. Like a mystic dragon, it mercilessly burns out the silicon devices to ash if those are not prepared enough (RAD-hardened). :-)
So, is it really true? What is the average time to a complete failure for a regular off-the-shelf component, like a microcontroller or memory? What is the period between, for instance, consecutive single upset events (once a day or once an hour)? And are there any chances to watch over them using software techniques only? For instance, by the periodic check of the ROM and RAM content?