Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
-
I don't use unit tests in embedded, and I won't change that. A lot of embedded code is about external behaviour, not computation. That's why hardware-in-the-loop-tests need to be done, typically with mocking up the whole environment, and then requirement based testing.
And no, TDD doesn't lead anywhere. Test design and test implementation shouldn't be done by the firmware coder at all so that independence is preserved. -
@Fast-Nop I do not disagree regarding hardware in the loop. That is most definitely needed.
In the automotive industry we per the OEMs we have need to follow ASpicd, v model and iso26262... if we don’t unit test we will fail the audits.
Do I believe we have the time to mock the entire system no. But we as the software always are the last to finish our portion because we are then last to receive physical hardware from the Electrical as they have to wait for mechanical group to finish.. and it all trickles down the line.
In the time we wait we know what the requirements are we have an idea of what needs to be done we do have some time to mock up units. Do we always succeed and do it right now because time.. so we compensate by the hardware in the loop tests.
The problem with this is it leaves a lot of room open to forgetting to tests functions for bound and ranges and valid inputs and outputs. This is where unit testing the functions either mocked on host or on target using Cmock or VectorCast or IBM concert I Believe it’s called? I don’t remember.
Anyway TDD.. is it the correct approach ? No we cannot follow it to a T I did not mean that.. but ideas from TDD can be successfully implemented in embedded systems to reduce problems found either at with HIL tests or god forbid things found in the field.
I’ve seen too many times where due to misreading of datasheets of ICs of the system leading to their full values being read by the system and acted upon are not fully covered. -
@Fast-Nop Example a light sensor receiving 0 from a light sensor and not covering a situation that low light.. because you assumed light level conditions in the environment could not get that low. Which they truly can’t. But you decide to make a function that does math on the result from the light sensor... that algorithm has division, and thus you end up dividing by 0 .... at first glance when writing the program you don’t see a problem because “the light level can never be that low” so we would never receive that value from the light sensor.....
You do your HIL and no issue becuase you/ testers designed the tests around normal conditions or slightly outside normal conditions maybe some extreme conditions but never do you simulate what happens if you receive a literal 0 from the light sensor? ... all your tests were from the normal and extreme values comming from the sensor.. AND you covered the condition that if no light sensor is present contain works aswell...
What you failed to realize 0 is an actual valid result from the light sensor and is used to tell that there is an internal error with the light sensor, it’s an error but valid result. Thus because the way it was written the person took data from the light sensor did not filtering and divided by zero causing undefined behavior.. a simple unit test would have caught this.
Will it catch everything no.
Is it becuase of programmer error yes.
Is everyone perfect no.. so to aid us.... unit test. It helps ensure all algorithms, functions etc function the way we expect prior to HIL testing and system integration.
Does programmers need to be abstracted from the test to ensure non bias yes, but we can write our own unit tests.. even better pair programming one writes the tests and the other writes the code. And flip flop. We started practicing that about a year ago and we have caught a lot more “stupid” run time errors that are completely independent of hardware and 100% todo with logic. -
@QuanticoCEO Well, we mock the entire system, and we don't have schedule problems (ok, we do, but for other reasons) because code implementation and testing implementation are done in parallel by different people.
On the other hand, we do have advanced tooling for static code analysis before the code even gets tested.
And in my private project, I have mocked that up so far that I don't even need any MCU hardware for testing the application layer. ^^ -
@Fast-Nop we use QAC static analysis as well as vectorcast for system integration, and unit testing and we use CANoe for communication testing.
Related Rants
-
From the guy who wrote all the Programming Microsoft books and the Annotated Turing book. Comes this book.
Th... -
Forgot to post a book yesterday, so maybe I’ll post two books today...
Anyway, this book, I found it recen... -
From NAND to Tetris..
This book is IMO the best book for those who want to venture to the lower level program...
Working in the embedded systems industry for most of my life, I can tell you methodical testing by the software engineers is significantly lacking. Compared to the higher level language development with unit tests and etc, something i think the higher level abstracted industry actually hit out the of park successfully.
The culture around unit testing and testing in general is far superior in java and the rest.
Down here in embedded all too often I hear “well it worked on my setup... it worked at my desk”.. or Oh I forgot to test that part.. or I didn’t think that perticular value could get passed in... etc I’ve heard it all. Then I’ve also heard, you can’t do TTD or unit tests like high level on embedded... HORSESHIT!
You most definitely can! This book is a great book to prove a point or use as confirmation you are doing things correctly. My history with this book was I gonna as doing my own technique of unit testing based on my experience in the high level. Was it perfect no but I caught much more than if I hadn’t done the testing. THEN I found this book, and was like ohh cool I’m glad I’m on the right thought process because essentially what they were doing in the book is what I was doing just slightly less structured and missing a few things.
I’ve seen coworkers immediately think it’s impossible to utilize host testing .. wrong.
Come to find out most the of problems actually are related to lack of abstraction or for thought out into software system design by many lone wolf embedded developers.. either being alone, or not having to think about repercussions of writing direct register writes in application or creating 1500 line “main functions” because their perception is “main = application”. (Not everyone is like this) but it seems to be related to the EEs writing code ( they don’t know wha the CS knows) and CS writing over abstraction and won’t fit on Embedded... then you have CEs that either get both sides or don’t.. the ones to understand the low level need but also get high level concepts and pariadigms and adapt them to low level requirements BOOM those are the special folks.
ANYway..the book is great because it’s a great beginner book for those embedded folks who don’t understand what TDD is or Unit testing and think they can’t do it because they are embedded. So all they do is AdHoc testing on the fly no recording results no concluding data very quick spot check and done....
If your embedded software engineers say they can’t unit test or do TDD or anything other than AdHoc Testing...Throw the book at them and say you want the unit test results report by next week Friday and walk away.
Lol
rant
bookaday