FlashFS source gone in release 4.20

I think Keil staff are busy shooting themselves in their feet with really big guns right now.

For some reason, they don't seem to realize the very critical need to have access to source code.

We can normally live with not having full sources to the C RTL, since in case we get into troubles we can find many free implementations of specific features.

While most companies would feel comfortable designing trivial things like lamp timers without full source code to target-specific functionality, more complex or critical devices will require source.

What company wants to invest many millions into critical while not being sure that they have access to the source code so that they can produce over-night critical hot-fixes if needed. Just failing a weekly shipment of 10k units could cost huge amounts of money, besides the goodwill losses from not being able to supply units to large and very influential customers.

And in some situations, a notified body - or a huge customer - may require that they can come and visit and look at development documents, source code, testing, fault processing routines etc. When human safety is involved, it isn't enough to tell that Keil has the source code and that there are unknown source code changes between two Keil releases.

I haven't upgraded yet, but if this is true then it pretty much makes upgrading a waste of time for me. It was bad enough not having the TCP code and having to rewrite feature after feature due to simple problems with keils code.

The original FlashFS code was basically very bug ridden when I started using it during version 3 something. I spent years helping Keil debug it through reporting errors and actually writing some of the fixes myself. I must have reported 8 serious problems from simple case sensitivity to full scale FAT corruption.

They also claimed FlashFS was thread safe when accessing FAT but this was not true through their own C libraries, (see my posts from the past) - this problem still applied to the version before 4.20. You have to put your own locks around every file system access to fix their problem between ARM's libraries and FlashFS.

How on earth can I trust their hidden code when I couldn't trust their visible code!

We have started porting alternative TCP stacks, and for now I think I will just stick with their old FlashFS code as it at least it mostly works and I could fix this myself. I fully expect the new version to have countless problems as no one seamed to perform any basic testing on the originally FlashFS before they started selling it!

Before I ported my code to ARM, I used uCOS-II under Keil, I wish I had stuck with it! I might even look at going back.

Anyway one thing is for sure is so long support contract as I was only buying it for updates to RL_ARM.

Keil have totally lost the plot.

Stuart,

Thanks for your post. There is a systemic problem here but Keil don't get it: the last time I ranted about these and many other very painful issues with FlashFS and indeed, RTX (about a week ago), I got an attitude (off the forum - it seems that I annoyed senior managers at Keil/ARM - was I impolite? I don't think so. Just stated some facts). But there is a problem here and Keil must understand and address this. Speaking of reputation, do you know that some of these problems tarnished/are damaging our reputation in the eyes of some of our clients, too?

Tamir,

I fully agree there is a problem.

However for years they have not understood the problem we have all had with not having the TCP code. Next I fully expect we will lose the source to RTX (I think this still remains in 4.20, but what about the next RTX version?).
They don't understand that debugging an (at times poorly tested) blackbox is a pain. Maybe they actually want to charge more for support?

Funny thing was at times I have felt like it was me giving Keil support for FlashFS! My boss actually suggested we investigate charging them for some of the problems as it had been so badly tested.

How can Keil compete with RT vendors that supply the full source code? Why would anyone pick Keils solution now? (The only advantage I can think of is the integration to the environment.)

In fact I can say that we initially thought the complete RL-ARM source code was included when we picked initially RL-ARM over uCOS-II (see my other posts) So when we discovered the TCP code was missing we were actually surprised/shocked! If it was not for this oversight we probably would never have picked RL-ARM in the first place!

I am now certain it was a mistake in going to RTX, one that we intend on now rectifying.

Stuart.

Stuart, all,

I have opened a case at ARM asking them to have a look at this thread. I am certainly not going to become emotional about this matter all over again; I don't think they are going to listen or change anything at all either way. Let me just reiterate this: there is a lot at stake here, Keil - not just your good name, but also ours!

Probably anyone with any embedded experience has spent countless hours debugging a problem that creeps up once and hour, day, etc. It may be my problem, a co worker’s, it may be the compiler’s, a hardware design problem, or other defective widget. The reality is that it does not matter who’s fault it is, the user has to determine what the source of the problem is. If it is compiler’s, we have to be able to demonstrate to the compiler vendor that they have a problem. They then decide if it is something that they want to fix or make the users work around. It is rarely simple enough to point the finger and say my project does not work, we think it is your problem please fix it ( though recently the forums have been used for this). The problem has to be identified to a line that compiled incorrectly, a function that behaves differently with optimization changes or compiler updates, etc. This process is time consuming enough with full source code. The time necessary to debug ARM disassembly code is prohibitive.

For unexpected conditions such as what happens when X occurs? Say this is power loss or card removal. I could not reliably exactly reproduce this with any by turning off power or yanking a memory chip or card. With the source, I can simply put a break in at this point and know where the problem was introduced.

After submitting several Keil trouble tickets, the process frequently goes like this:

1) Go to tech support web page. In 1000 words or less, explain the problem.
**Note there in no place to attach an example project.**
2) Get reply with call number and request for sample project.
3) Send in project or sample project.
4) argue for a few days as to if there is or is not a problem.
5) finally agree that there is a problem.
6) wait for a fix or work around.

This is the reality of the process. One cannot necessarily blame the process as the number of invalid requests must be quite high. (#4 has been reduced with an increasing number of valid call requests.)

Of the request tickets most were simulator related and many have eventually been fixed. These are much more understandable than code generation problems of which I have reported 1 (it has been fixed) and have another to report when an example gets made. Both code problems are quite similar and involve using a float as an argument.

So, what would the ticket process will be for “my FS does X once every 5 or 10 days” when we argue about things that can be demonstrated in 2 minutes by changing optimization levels?

Is Keil going to step through their locked up code for us and find the problems regardless of fault? If so, perhaps a thank you is in order for saving us the debug time.

We chose Keil as the compiler as a result of the unique simulation capabilities and would not want to change. There is, however little special about binary only middle ware.

We have the word back from Keil that the source to FlashFS and USB have been removed in V4.20. As per Chad's comment above, I have also found many occasions when being able to step through the middleware solved problems. With the source, you can perform self-rescue, or see where Keil did something you were not expecting. I can se Keils' support requests heading skywards!

If you bought a bicycle with a tire pump, took it back to the shop for a service, the shop remove the pump on the basis you dont actually need it to ride the bike. Low and behold riding it home, you get a puncture and have to wait for someone to rescue you (Keil support to get round to answering support request among the thousands they undoubtedly will now get, while the chief is yapping at your' heel wanting the problem fixed *NOW*).

- Do we get a refund for the reduction of package content?)

Like many others, a few years ago when we bought RL-ARM 'with source code', it was not stated that source to TCP was not present. I was burnt and embarrassed by that oversight, now to remove more - there is little reason to continue the annual support on RL-ARM.

- Phil

I agree very good compiler, but with poor to average (now) binary only middle ware.

Somehow I suspect the number of support calls at the "4) argue for a few days as to if there is or is not a problem." stage will increase significantly as its been hard enough convincing Keil there is a problem even when you can actually point to the exact line or function!

Phil,

I think it was established in the prior thread Tamir mentioned that the refund on the price for this significant reduction in package content is actually a highly generous price increase!

I was actually laughing as I wrote that - as it just seems so insane|

Basically it was expensive enough when we had the source, although as you say we did all believe we had the source to TCP stack when we first bought it a few years back, I wish I had realised back then how this would go and got out asap. I would never advise anyone to use RL-ARM middleware now.

Stuart,

I have removed MDK 4.20 last week shortly after installing it as it imposed too many structual changes to existing products, so I haven't noticed all the omissions - but do you mean that they actually removed the hardware drivers of USB (usb_hw.c etc.) ?

One can only wonder what has prompted Keil to start this foot-shooting frenzy. We've been using Keil tools and middleware for some years now, but I have never felt that the products have been worth the (very high) price. The only reason we're using the uVision IDE is that occasionally we need to use the integrated debugger. The project/target management is a joke, and file grouping is so limited it is of no use.

The compiler itself has been good (except for one bug in a recent version that actually broke our code), and RTX has (in general) served us well, but these moves towards closing their middleware sources will force us to consider whether we should take a different path in the future.

I am pretty sure that Keil are relying on the fact that moving away from them comes at a cost, and that most customers therefore will choose to live with the changes. Keil will only reconsider if a large number of customers leaves them, but I'm not sure that is going to happen.

I took a brief look at uC/OS-II, and my first impression is that seems like a well-documented RTOS, and with a lot of choices when it comes to additional middleware, like IP/USB/CAN etc.

Øyvind Kaurstad,

I see it as an exercise in long term/short term planning/economics. I work for a small company that does not have the financial means to maintain the current arrangements with Keil as they are: Some budget was made available for the procurement of software licenses at the beginning of 2011, and we cannot exceed that budget - period. We rely mainly on the skill of our staff - development tools should only play a side-show (in my opinion).

No I didn't mean they removed the drivers, I would really hope the drivers are still there but I haven't installed version 4.20 and I probably never will now.

However you could look at it this way what is stopping Keil removing the drivers? After all Keil doesn't seem to understand or even listen to their customers over requirements to access the source code. My trust in Keil has totally evaporated.