Brief version: How can I debug my Chromebook woes when trying to run kernel-next with the Mali drivers?
I followed this guide: Graphics and Compute Development on Samsung Chromebook It worked, but I'm left with an incredibly unstable system. I'm running a Samsung Chromebook, model number SNOW FREMONT A-E 0878.
A list of stability issues so far:
I'm even sure there's more, but for now that should give a good idea of what's up: ridiculous instability. I might be able to throw the image on a USB drive and use a USB wifi adapter and skate by some of this, but that's really not a sufficient solution. Given that and the incredible unlikeliness that anyone here has already had or is having this issue, I'd really like it if someone could tell me how to debug these issues as I don't have any clue how to approach it. I have a bit of familiarity with the Linux kernel and a ton with C, but not a darn clue how I'd go about debugging Linux running on a Chromebook.
Hope this is related enough to be discussed here - I certainly don't know where else to discuss problems with this guide.
Hm, could it possibly be related to what kind of SD card is being used? I'm using a Sandisk Pixtor 32 GB Class 10 SD card. I ask because I vaguely remember similar problems with running the Raspberry Pi device with a greater-than-Class 6 SD card (although I did not experience them myself.)
I'd like to test this hypothesis, but I don't have any other free SD cards on hand. I bought this one specifically to do this guide, in fact.
I've seen problems with specific SD vendors not playing nice in the past with certain boards, so it wouldn't surprise me! For what it's worth, I use a mixture of Sandisk 8GB SDHC class 4, and Sandisk Ultra 8GB MicroSDHC class 10, and I have SOME issues on the former but as long as I don't do anything too taxing, i.e. boot and immediately mount NFS/USB then it's fine. Maybe someone out there can fix and push upstream?
So Chris are you saying that you do have the problem with some SD cards and not others? I'm having the problem with the internal mmc itself.
From what I can tell, it may be a combination of both. I've migrated over to USB for the time being, which works, although the USB stick is pretty crappy. However, Wifi still occasionally gives out, and in fact, you're right: I can still see the internal mmc crashing once in a while, even though I'm not doing anything with it.
I don't know if I have enough free-time to look for what the actual issue might be. Part of me wants to try jumping back in kernel-next and trying to see if the issue still occurs months ago, but then I don't know what patches I may need to leave in for the ARM Chromebook to still boot properly.
Hi fattire,
Sorry, misread your original post, I do see some instability sometimes on certain SD cards, but generally it doesn't stop me mounting nfs and testing things over that. I only really use the SD card for booting. I haven't tried using internal mmc but its surprising that doesn't work. I'll feed back these issues to the relevant team so they can look into them for a future release.
Thanks,
Chris
Hey Chris,
It's been a couple months. Any words on fixes for those of us with internal emmc issues?
Update: I spent a little more time looking at the crash. It's so totally random-- can be while booting, but can also be after 3-5 minutes after logging in.
Since the panic doens't seem to write anywhere, I grabbed a screenshot to see what was up... looks like s5p_mfc_init_pm was failing to get a clock-gating control, whatever that means. I found this, and applied it, along with this to the 3.8 kernel (had to tweak a little), but didn't seem to do anything.
Do i give up? Someone out there must know what to do...
I've let our team know you are still in need of some help here. In future if you can make a discussion as a question it will ensure it gets more visibility as unanswered questions remain at the top of the homepage. I've moved this to a question now so you will be able to select an answer as correct and helpful once this has been resolved.
Ryan
No update from our side, my only advice would be to use USB as this is more stable.
The kernel we are using (3.8) is still considered unstable, 3.4 being the stable kernel, but we decided to use 3.8 for performance reasons. 3.4 might fix this problem but wouldn't be officially supported by us, and would likely result in significantly reduced performance. We are strictly speaking just responsible for the Mali side of things, so while we do have a few kernel hackers on board who are aware of the problem, I can't say that we WILL fix this problem, and ultimately the fix would very likely have to come from elsewhere in the community, but it is at least on our radar and we are keeping it in mind in case we can fix/work around the problem. If the problem is not present in 3.4 maybe someone can take a look at the differences?
I've had a quick look and the instructions and they refer to a specific commit on an old repo, which has now moved and is active (last commit today!). It might be worth trying with HEAD on that, in case the problem has been solved? We will test with this and check if it's stable on our side but you'll be better at telling us whether this has fixed it for you
Can you explain how you got the system to boot from a USB stick? I built as per the development guide with a 16GB usb stick as the target and the boot hangs once the it detects the stick.
--Mike
Heya-- thanks for the response-
Looks like after the 3.8.11 tag they moved on to 3.10.. [see below-- you're talking about the chromeos-3.8 branch...] . I'm tethering off my phone right now so don't wanna grab the latest source, but I can give it a shot in the next few days and get back to you. I hope they checked something in. I don't know if I mentioned in another thread I did try applying the changes to mainline kernel at HEAD (as recommended here), but there were SO many changes-- code didn't match up-- some of the files didn't even exist any more. I started hand-checking stuff and matching up moved headers and adding files and such but it quickly became a rabbit hole, and I gave up after a few hours.
I'll try kernel-next's HEAD later and see what happens, and report back here. I have a feeling though the source will be too different and/or the Samsung Chromebook has been long abandoned by the Google engineers... but I'll letcha know.
Thanks! If anyone else wants to try too, lemme know.
Update-- oh you mean this branch chromeos-3.8, not the raw v3.8 branch, huh? I'll try this. Also, I'll update this thread as I go... hopefully it doesn't send a new email to you every time I change it... if it does, let me know.
Update #2: Build breaks here:
drivers/gpu/arm/t6xx/kbase/src/platform/chromebook/mali_kbase_config_chromebook.c:376:2: error: unknown field ‘power_suspend_callback’ specified in initializer
.power_suspend_callback = pm_callback_suspend,
^
drivers/gpu/arm/t6xx/kbase/src/platform/chromebook/mali_kbase_config_chromebook.c:377:2: error: unknown field ‘power_resume_callback’ specified in initializer
.power_resume_callback = NULL
I commented both of the above out and the build went to the end, though I did see some "arch/arm/boot/compressed/head.S:978: Warning: (null)" when making the zImage.
Next, in kernel.its, "arch/arm/boot/dts/exynos5250-snow.dtb" doesnt' exist any more.. so I guessed and used arch/arm/boot/dts/exynos5250-snow-rev5.dtb
The rest of the kernel built normally and I have some kernel modules + a boot.img
When booting, I get a Oops 5.... mmcqd in the same place-- dw_mci_pre_dma_transfer.isra.... from dw_mci_pre_req... again at random times... sometimes on boot, sometimes after UI starts... about 15 seconds. So not fixed... it actually seems worse as I'm trying to get in to at least boot back to chromeos by default... :/
(mmc2 / wifi having issues too, but I'm not worrying about that...)
Any suggestions would be appreciated.
This seems to confirm the mmc issue is still present thanks for trying. A lot of the wifi issues we've seen are userspace related (outdated udev) so people having trouble could try using a more recent distribution, people in the office are using Arch for example without issue.
mikewinter: I haven't actually got it BOOTING from USB, sorry to not be clear, I meant to boot from SD card and as soon as possible move to using USB stick/NFS mount for actual work, as this is far more stable.
Yeah it seems to be there still unfortunately. I thought for a second it may have been related to interference from the USB (which was plugged in, and from which I had flashed boot.img) but even after pulling it, though it didn't crash as quickly it seemed, it still crashed..
The outdated userspace stuff makes total sense... if this one bug were fixed I'd probably be rockin' and rolling in no time... so strange that the new 3.8 branch doens't have something so basic fixed... I wonder if it's some kind of interactiion with the changes I made... I dunno.
If it would be helpful to anyone I can put the 3.8 kernel + the patches from arm + the new device driver on github just to save some time...