Home > On-Demand Archives > Q&A Sessions >
Live Q&A - Buried Treasure and Map Files
Elecia White - Watch Now - EOC 2021 - Duration: 20:09
Useful presentation and Q&A.
I mainly use them to paste the usage summary at the bottom into Excel, to plot memory growth over time. I can then extrapolate and see visually if it?s going to have sufficient space by the end of the project. Occasionally it has shown it?s getting too tight, but after shuffling things around in memory the new slope and offset of the line shows that it?s back on track, with plenty of future growth potential. I store all mine as artefacts with every Jenkins build, so you don?t have to rebuild an old version just to get this information. I need to script this sometime, but it takes about 10 minutes every few weeks to keep Excel up to date with the last few builds I?ve done.
The comment from Michael (sorry if spelt incorrectly) about the bootloader was interesting. Were both linker command files set so they didn?t overlap in memory and the build would simply fail if it ran out of space?
Only 1 minute into your presentation and I realized that we are kindred spirits! :) I totally identified with what you said about the joy of eking out a few more processor cycles, efficiently writing a hardware driver, or reducing memory consumption! I'm glad to hear that there are still other people out there that enjoy this.
This was an excellent presentation!
Great job Elecia of making map files beautiful! Your map art has a "Lord of the Rings" feel to me so the only thing missing is in one corner your portrait dressed up as Galadriel Elf Queen of Memory Map Land. :-) A couple of comments: To extend the Analogy the Linker Definition file is the Cartographer. I have quoted you in the past for your definition of an Embedded System from your book "An embedded system is a computerized system that is purpose-built for its application." I would like to submit a possible test if someone is an experience Embedded System Engineer is "Have you used a map file to find and fix a defect". I highly recommend anyone who is an Embedded System Developer listens to the Embedded Podcast https://embedded.fm/ and help support by being a Patron with the side benefit of getting on the community Slack Channel.
It's an interesting idea to use the map-file for analyzing the firmware size. Most of the time I was experimenting with "bloaty" (https://github.com/google/bloaty). Although this tool is parsing the elf file and not the map-file. Maybe also interesting to look at.
Thanks for the great talk and some cool insights into map-files.
That's a very cool memory map! I love it. Just one thing that may sound like a nitpick, but might confuse people with map-making experience: Rivers never split. So when I saw the .cinit area, and the river of initialization, I immediately though that the .bss influenced .cinit, not the other way around. A better way to represent what you want would probably be making .cinit be surronded by 2 rivers, one going to the ocean, the other to the lake.
regardless, that map is a huge help, thank you for sharing it!
Your map of the map file was very clever. I might have to find a space on my wall for it. Very well done presentation on a formidable topic. I found map files inscrutable initially and struggled to make sense of them, but over the years the information they contained was invaluable in debugging.
Nice Job Elecia, That colorful map is a great way to highlight what a .map file is all about. Really makes the point.
How can I access "the map"? I reeeeally need to print and frame it. For work.
My website has the HD image (click on the map):
http://embedded.fm/blog/MapFiles
And when I asked a graphic designer friend where they should be printed, Society6 was the recommendation so I put it there (with a 1% markup, the minimum I could do). So if you want to print it elsewhere, go ahead. Maybe send me a pic of it in your lab/office/above your fireplace.
https://society6.com/embeddedfm
Hi Elecia, that was a great talk!
I still remember having to troubleshoot missing startup code (no zeroing, no initialized variables) in my final uni project. This taught me the answer to that eternal interview question of "what happens before main".
How often have you had to write or adapt your startup code in real-world projects?
Some... but less in the last few years. Now that most systems have a startup.s and weak bindings for the interrupt vectors, startup code is more canned... that said, one area I used it on recently involved reading constants (specially marked section) from the spi flash into fast RAM for graphic assets for boot time imagery.
Interesting, do you simply modify the startup.s and call your own function to add your code, or how do you add it to the startup?
That's an elegant solution, will keep it in my toolbox
Oh, that reminds me. I created a small hobby project on an 8 bit avr about 6-7 years ago, written in C++. My first, trivial hello world-type code loaded and worked fine. But very soon when adding a bit of code, everything would compile, load fine, but the micro just seemed completely dead after loading. With no debugger I tried to light an LED in line 1 of main, nothing. Having no better resort, I used objdump to get a disassembly of the code and in part of the pre-main startup code linked in by avrgcc I found that a trivial file would do some memory clearing and then jump to another piece of initialization code, but with my non-working code, it instead jumped miles outside of any available memory. It was in a function called something with "ctors", so on a hunch I tried moving the one global object I had into main instead, so that it got allocated on the stack, suddenly everything worked perfectly!
For address/symbol lookup you can also use nm and addr2line (at least in the gcc toolchain).
addr2line, very nice when you're running a debugger in an environment that doesn't have access to the sources. A quick one-liner immediately tells you where in the code you are.
I'm going to add JeanLabrosse's suggestion of elfread as that seems quite useful.
Definitely, I believe the name is readelf by the way.
Saw that Embedded.fm is on YouTube. Will you be doing video podcasts and interviews? Thought you could be very creative with them after seeing this presentation.
Probably not... Video is a really hard medium. It would require too much time so we'd have to make money from it but I'd rather share information freely and get paid to do engineering.
Any plans to produce the map as a print we could buy? :)
Sure! My website has the HD image (click on the map):
http://embedded.fm/blog/MapFiles
And when I asked a graphic designer friend where they should be printed, Society6 was the recommendation so I put it there (with a 1% markup, the minimum I could do). So if you want to print it elsewhere, go ahead. Maybe send me a pic of it in your lab/office/above your fireplace.
https://society6.com/embeddedfm
Thanks Elecia, and thanks again for the presentation. Loved the style and take on what could have been a real snooze fest! Hope to see more presentations in the future.
You should definitely make it into a board game. "Come on kids, let's play hard fault!"
It's available here as it seems: https://society6.com/embeddedfm
You can even change the currency. It would be cool if the ? price means prints are shipped from within Europe :)
Awesome, thanks Tarik
Hi Elecia! Great talk! Long-time-listener-first-time-caller. Gcc and other compilers have been around for a while now and I've wrestled with them (and linker scripts) myself. Why do you think these files are still so difficult to read? Do compiler vendors not realize that they could help developers a lot by making them more user-friendly?
Not enough people ask for them to better... It is one of those areas that isn't as spiffy as 2% faster compiles or better security for their licenses.
Very original. Loved "the map"!
Thanks!
Do you like to 'name sections' to allow you to determine where your section is located or do you simply always let the linker place memory? Naming sections is important if you use an MPU but not as critical for non-MPU systems.
Naming sections is also important if you have something that needs specific placement: bootloader, non-volatile storage, config information to pass between images. But if I don't have to, I'll let the linker do the work.
16:40:58 From Rocco Brandi : Elecia's book should be a mandatory reading in colleges and universities! 16:41:06 From Leandro Pérez : Is amazing your way to explain the map file... Flash Federarion, Ram Landia and Kindgdom of SPI Flash... Amazing!!!! I have surf in the map file of the time to solve RAM and FLASH problems related with not enough space 16:41:09 From Brian Schmalz : Nice music! 16:41:21 From Leandro Pérez : I have a question... What resource you recommend to deep into the MAP files? Maybe your book? 16:41:22 From James A. Langbridge : I actually let my students read my copy (signed by Elecia, with a very kind mention, thank you!) 16:41:31 From Jay : I am here for the pirate jokes 16:41:33 From Michael Kirkhart : https://embedded.fm/blog/mapfiles 16:41:56 From Tim Michals : Now I'm thinking of writing a MAP viewer and compare tool using electron. 16:42:01 From Tarik Kurtovic : Elecia's book is in my team's library ;) not forced reading but new members do get a nudge in the right direction 16:43:07 From javi : STM32CubeIDE creates linker scripts for STM32Fxx devices that contain .init and .fini input sections as well as output sections like .preinit_array, .init_array and .fini_array. I know these kind of sections are used for C++ static objects constructors that are called before main and destructors called after main, but are they used for anything in C? 16:43:18 From James A. Langbridge : Stuck in the past? I feel old now :( 16:43:19 From Alex Burka : With an RTOS like FreeRTOS, you can have stacks and other stuff that don't get allocated until runtime. Is there a way to use these techniques to debug that? 16:43:25 From Mikael Albertsson : @Elecia, did Chris create the music? =) 16:44:24 From Anthony Koutroulis : What software did you use to make the (illustrated) map? :D 16:44:34 From Thomas LeMense : Excellent presentation, and beautiful map! Are you aware of map visualizers available that interpret the (apparently somewhat standardized) linker script language used by ld? 16:44:40 From Leandro Pérez : Thanks @Elecia 16:44:42 From Elecia : yes, Chris created the music and did all the video 16:46:08 From Erwin : Hello Elecia, thanks for this amazing way to explain map files. I really love this cool map! As an embedded engineer you should really learn to read map files! 16:46:36 From Tim Michals : @javi you can use the init to call C functions before main, i.e. initializations 16:46:38 From Gerhard : @javi: .init/.fini might be used, if you use the constructor/deconstructor function attributes: https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html#Common-Function-Attributes 16:46:48 From Will Hsiung : Just saw your YouTube channel. Will you be making video episodes and interviews as well as audio? Thought your presentation is cool, especially with the illustrated map! 16:47:56 From Sam : Thank you for the presentation. So far, #1 on the presentations in this CON! Loved the fact that you went deep. Can you take apart printf the way you were showing? 16:48:41 From Sam : What is a magic number? I built a small Linux OS and that was in the assembly code. 16:48:59 From javi : thanks Elicia, and thans Gerhard! : ) 16:50:08 From Sam : odd question...were you wearing a USB stick necklace in the brief? 16:51:25 From javi : Thanks @Tim 16:51:51 From Anthony Koutroulis : @Sam, As I understand - magic numbers are literal constants that don't reveal any significance by their value, but are significant to the code. Like register values without #defined names 16:52:09 From metty : I like the octopus under the ocea 16:52:14 From metty : n 16:52:19 From Michael Kirkhart : One example (not tried out): https://github.com/PromyLOPh/linkermapviz 16:52:44 From Sam : yes 16:52:47 From Sam : thank you. 16:52:56 From Gopinath : Just a comment. GNU binary utilities have a bunch of useful tools to get info from an ELF file, which is also in the map file. 16:52:56 From Raul Pando : My understanding about magic numbers is the same as yours @Anthony 16:54:15 From Tom Tom : Are there any micros, or tools, that you'd consider icky and avoid like the plague? 16:54:54 From Jeremy Schreiber : Hi Elecia! Great talk! Long-time-listener-first-time-caller. Gcc and other compilers have been around for a while now and I've wrestled with them (and linker scripts) myself. Why do you think these files are still so difficult to read? Do compiler vendors not realize that they could help developers a lot by making them more user-friendly? 16:54:57 From Michael Kirkhart : @ Mikael: Great example of using the map file for memory size debugging 16:54:58 From Tim Michals : One of the best examples of using a MAP file. Could not figure out why accessing SDRAM was so long for a data structure. Overlayed the memory onto SDRAM pages, the issue was each page requires a refresh, so updated the structure to access data within the page 16:55:08 From Piotr Zdunek : Hi Elecia! I'm a big fan of the podcast :) My question is about embedded software quality. I've worked in the industry for several years and I see that management doesn't care about it, what do you think is the best way to improve it? How to convince the management that quality is important? 16:56:39 From Gerhard : For symbol location/sizes: in a gcc toolchain you can also use nm -S your.bin 16:57:35 From Tarik Kurtovic : Hi Elecia! Thanks for the map, which is the most interesting way to present this topic I've seen to date. In one of my first concrete experiences with map files and embedded software I ended up also writing the startup code, helping me understand where zeroing and initialization comes from. How often do you have to do something like than in your work? 16:57:39 From Rob Meades : Though readelf probably only works if you compiled with full debug info -g3. 16:57:45 From Tim Michals : https://man7.org/linux/man-pages/man1/readelf.1.html 16:58:57 From Mikael Albertsson : @Jean, good tip, readelf also has command line option letting you tell it what types of symbols you're interested in, meaning you don't need to write post-processing scripts to find the things you need to the same extent 16:59:22 From javi : In GCC, you can use -ffunction-sections flag to create a section for every single function. That way the linker can discard the whole object file but the funtions it uses 16:59:23 From Mikael Albertsson : Also, if you're working on C++ code, c++filt is a very nice tool in conjunction with readelf and map files 16:59:28 From Keith J : Thanks for taking the time to put together a great presentation Elicia! 16:59:36 From enrico : LIGHTNING ROUND 16:59:43 From Sam : thank you. 16:59:50 From Rob Meades : Thank you, that was way more fun than I though it could ever be. 17:00:07 From James G : Thank you Elicia! Wonderful talk, learned a bunch from just a short video! 17:00:10 From Gerhard : Thanks! 17:00:16 From javi : Thank you Elecia, one of the best talks of the conference 17:00:16 From Raul Pando : Thanks a lot Elicia, great work as always 17:00:19 From Thomas LeMense : Thank you!! 17:00:19 From Gopinath : Thank you Elecia. Great presentation. 17:00:19 From Leandro Pérez : Thanks you @Elecia 17:00:22 From Mikael Albertsson : Turn it into a board game! 17:00:24 From Tony Arkles : Thank you! Always a treat Elicia! 17:00:24 From Leandro Pérez : Amazing presentation 17:00:25 From Jean Labrosse : Great talk Elicia! 17:00:26 From lshi : Nice Map fugruation thanks you 17:00:28 From Will Hsiung : Thanks for the presentation Elicia! 17:00:33 From Erwin : Thanks a lot! 17:00:46 From Tom Tom : Where do we buy a print? :) 17:00:53 From Will Hsiung : *Elecia! 17:00:58 From Tony Arkles : *Elecia :D 17:00:59 From Leandro Pérez : lol 17:01:03 From Jean Labrosse : Sorry Elecia 17:01:14 From Keith J : whoops 17:01:17 From metty : Thanks 17:01:22 From Mikael Albertsson : Thanks!
That map is so cool I'm definitely going to print it and stick to my office waall!