Friday, March 30, 2012

Testing Feedback Needed: Android 4.0.4

This is NOT A RELEASE!  This is our first working build of 4.0.4 and with very little testing so it may have hidden breakage.  Please test only if you plan on providing feedback to help us improve before CM9 beta2.  CWM backup is highly recommended!

Help Needed - Advanced Users Only
CM9 beta1-test-4.0.4 is CM9 beta1 with Android 4.0.4 built on March 30th, 2012.  The audio fix is not ready so it is not included in this build, although mdnie is enabled.  Please help by testing this ROM and reporting any of the following.
  • See anything different from CM9 beta1?  These are not bugs but just changes in behavior.
  • Any regressions since CM9 beta1?  These are ONLY new bugs.  If something worked in CM9 beta1 but no longer in this build, we need to know about it.  Important: Please post your feedback even if it is the same bug as someone else's report.  We need multiple reports to better confirm new bugs!
Warning!  3rd party themes are known to break this ROM.  Do NOT use 3rd party kernels with this ROM or your test feedback may confuse the developers.

Download CM9 beta1-test-4.0.4
Mirror [1] [2] [3] [4] [5]
md5sum b4543959168db320ea2c9f58f057c01b

Please reply here in the blog or here on Google+.  Your feedback will help us to improve CM9 beta2.  Thank you!

Monday, March 26, 2012

Kernel 3.0.x Boots! (Sort of...)

After weeks of effort, bbelos got the kernel-3.0.x to boot on the Epic 4G!

Well... nothing works.  No radios.  No camera.  No sdcard.  It also crashes very quickly.

Anyway, this is progress!  bbelos continues work on the kernel merge.  Still could be a few weeks before it is as stable as our existing 2.6.35 kernel.

UPDATE 3/27/2012: I forgot to mention, we are not 100% sure this is possible yet, but we are hoping the crespo kernel merge will allow us to use crespo4g (Nexus S 4G) proprietary blobs including the crespo4g modem and menus that allow Activation, Profile, PRL Update and sensor calibration. If crespo4g's modem works on Epic with this new kernel, then we will be able to provide a complete flashable Heimdall image to flash any Epic directly to CM9, and you would have no reason to ever go back to TW rom.

UPDATE 4/1/2012: bbelos wrote this status update on Google+.
Just a quick update on the Epic4g Kernel 3.0.x port:

What works
MTD/YAFFS2 partitions, Calls / texts, 3G Data, GPS, Bluetooth (activates, device pairing untested), Touchscreen, Softkeys, 
Keyboard (mostly), Reboot to recovery, adb

what doesn't work yet
Sleep issues (system is stable when connected to a pc), Audio, Wifi, sdcard

Anything not listed has either not been attempted to date or has just gone untested.

Sunday, March 18, 2012

CM9 beta1 for Epic 4G Released

The Epic CM team is proud to announce the release of CM9 beta1 for Sprint's Samsung Epic 4G. Please follow us on Google+, Facebook, or Twitter for the latest news!  This release contains bugfixes and a few minor features since beta0.
Tour of Android 4 CM9 on Samsung Epic 4G
Changelog of CM9 beta1 (see previous)
  • Voicemail Notification option for those suffering from the "Phantom Voicemail" bug.  Phone < menu button > Settings > Voicemail Notifications is disabled by default.  Epic users on non-Sprint or overseas CDMA carriers may want to enable this option if they want voicemail notifications.
  • CM9 boot animation version 2
  • Device Encryption kernel support enabled.
    • Currently Untested!  Not compatible with CWM! Do not use!
  • 1.9MB memory freed by disabling the FIMC1 mmap and reducing mmap usage for the JPEG driver.  It is unclear what if any negative effects this causes.  crespo is also testing this currently.
  • Disable Samsung's mdnie improves color quality.
  • New CM9 Features
    • Holding Volume Up/Down while screen is off will change the current playing music.
    • Settings>Profiles
    • Settings>Developer Options>Root Access allows you to choose your root access mode.
    • Settings>Display>Rotation now has additional rotation options.
Release Notes
  • Thanks to user feedback, bbelos decided to leave the Dock Audio behavior as-is.  Alarm and ringtones will play from the phone's speaker, not the dock.
  • The newly working Sprint Visual Voicemail (see Customization Guide) has a few glitches, the most annoying of which is that it does not block the text messages from 9016 that you receive when there is a new voicemail.  We will fix this eventually.
Known Bugs
  • When using a wired headset, users have reported that the microphone volume is too low, making it difficult to be understood on the phone.
  • These issues are targeted to be fixed by, or after the crespo kernel merge.  bbelos is making good progress on the kernel-3.0.x port.  Not booting yet but getting closer.
    • CM9 for Epic is missing 5 of the 8 motion sensors usable in Gingerbread.  This apparently is the cause of auto-rotation not working for the minority of users.  Many of these users are able to make auto-rotation work by calibrating sensors using the temporary sdcard EL30.  Thanks to Joseph Shugzda for identifying this issue!
    • Maximum speaker volume is much smaller than Samsung's ROMs.
Links
See the CM9 Status page for more details. Then proceed to the CM9 Install Guide.  Please check if your bugs are already filed in the Epic CM Issue Tracker.

If you appreciate this work, please consider supporting the Epic CM team. Thank you!

Wednesday, March 14, 2012

Help Needed: Test Kernel for beta0

boot_beta0_disable_mdnie_tempkernel.zip
md5sum 851618ac07a41b4ef074012fe43e0732

This is a temporary kernel with two of the experimental patches we are testing for the upcoming CM9 beta1.  You can boot it by "flashing" in CWM.  Nothing is actually flashed.  It just boots this alternate kernel and your phone behaves like normal except with these changes below.  When you want to switch back to the beta0 kernel, just reboot.  Your feedback will help to determine if we will keep these changes for beta1 or not.
  • Disable Samsung's mdnie.
    • I have no idea what mdnie does, but noobnl says it "makes the color look bad".  I personally think it makes some fonts look bad.  What changes do you notice?  Which do you prefer, beta0 or the way things look with this temporary kernel?
  • Free more RAM by disabling FIMC1 mmap and reduce mmap usage for JPEG driver. 1.9 MB is freed.
    • This has something to do with the camera, but we don't know what.  It seems to work.  Can you find anything that this breaks or slows down?
Please reply with your feedback here or on our Google+ page!

Monday, March 12, 2012

CM7 Beta 1 Released

Well folks, its that time!
Yes, for those of you still using good ol' CM7, there's a new version for you!
The Epic CM team is proud to announce CM7 beta 1 for the Samsung Epic 4g. This new build has very few bugs, and some fixes backported from CM9. This is the last chance to test before RC1!

Changelog
  • Radio Fixes, such as the 'Ring of death' issue and the "Call has been lost" dialog on a far-end call hangup
  • Audio fixes, which improve sound quality and allow better in-call volume changing
  • Disappearing sdcard ringtone fix. Changes to the media provider to keep ringtones from disappearing randomly.

Links
Check the CM7 Status page for download links and installation directions.
If you still have any issues with CM7, you can report them to the thread on xda.

Wednesday, March 7, 2012

CM9 beta0 for Epic 4G released

The Epic CM team is proud to announce the release of CM9 beta0 for Sprint's Samsung Epic 4G. Please follow us on Google+ or Facebook for the latest news!  This is an exciting new release as we now consider CM9 for Epic 4G to be of beta quality because all critical bugs are now fixed.  What remains are only annoyances or missing non-core functionality not yet implemented upstream in CM9.

Tour of Android 4 CM9 on Samsung Epic 4G
Changelog
  • 36MB memory has been freed!  Upgraded the MFC driver to match crespo, then found the minimum amount of cache necessary to allow camera, camcorder and video playback to work.  This allowed us to free 36MB memory from the kernel.
  • Fixed the annoying disappearing sdcard ringtone issue.  Thanks to everyone who contributed discussion and logs on the problem!
  • Fixed visible artifacts during video playback.  Youtube, Netflix and other videos should be less distorted now.
  • Fixed 720p video playback skipping caused by the video artifacts fix.
  • Fixed the signal bars to behave like stock ROM.  No real change in signal, it just makes users feel better about the signal bars and Time Without Signal percentage.
  • Dock audio kernel support.  Requires the Galaxy Dock Sound Redirector app from Market.
  • Fixed (one cause of) data reconnection failures that otherwise aren't remedied by toggling mobile data or airplane mode, but only by reboot.
  • Quiet Hours settings allow you to customize the behavior of notifications during the night.
  • XZ compression of kernel shrinks the ROM size by 1MB.
  • Improvements to kernel build and debug support scripts.  Not relevant to users, but helpful to Epic kernel developers.
    Known Bugs (The Most Annoying)
    • Gallery: There is a serious bug in the Gallery app where it thinks certain recent photos are from year 1969.  This affects both CM9 Gallery2.apk and Google's GalleryGoogle.apk.  We are trying to diagnose this problem.
    • Battery: Sometimes the camera is stuck on after you are no longer using the camera app, causing the battery to drain quickly even when the CPU is in deep sleep.  Until we fix this bug, please reboot the phone if you notice faster battery drain after using the camera.  Camera and Google Talk have been known to trigger this bug.
    • Battery: If your sdcard has many files you should add .nomedia files to all directories that do not contain media like photos or music so the media scanner will skip them.  You especially want .nomedia in directories containing CWM or Titanium backups.  For most people media scanning takes only a few seconds, but some experience 30+ minutes of 100% CPU usage during media scanning due to a rare bug that we have not yet been able to isolate.
    Links
    See the CM9 Status page for more details. Then proceed to the CM9 Install Guide.  Please check if your bugs are already filed in the Epic CM Issue Tracker.

    If you appreciate this work, please consider supporting the Epic CM team. Thank you!

    Customization Guide

    Updated 9/30/2013:This guide is a menu of options to customize your experience after you have installed CM9 on the Samsung Epic 4G.

    Battery Optimization Guide

    Updated June 2nd, 2012
    This guide gives recommendations on Epic 4G batteries, explains the many options you can customize to improve battery life and how to diagnose battery drain problems on CM9 for the Samsung Epic 4G.  Some of this advice is based upon our current understanding of bugs in CM9 and how to best workaround causes of battery drain until the developers are able to fix their causes.  We will update this guide as we learn more.  If you have suggestions for further improve this guide, please reply in the comments below.

    Tuesday, March 6, 2012

    Shipping Epic 4G to Spain

    Today we bought an Epic 4G from Craigslist for $60.  It works fine but the glass is cracked.  Tomorrow we are shipping it to Spain!  Makes total sense, right?

    Sometimes I wish I had a real camera...

    The CM9 crespo maintainer KalimochoAz will help us with our kernel-3.0.x port.  We will merge the Epic and Nexus S kernel into the same kernel tree, currently based on kernel-3.0.23.  Epic uses many of the same hardware drivers as crespo.  By merging our kernel source, we can more easily benefit from the combined efforts of our two sizable development and testing teams.  In the medium term we can explore the possibility of merging in other Samsung Galaxy S phones.  In the long-term, we hope to rebase the kernel to upstream kernel-3.4+ after Android mainlining is complete.  Perhaps we will one day be able to push our device drivers to the linux-next staging tree.  That is the vision.

    Meanwhile, bbelos will have our team's Nexus S 4G.  He will work with KalimochoAz on the kernel-3.0.x port.  That way both developers will be able to verify that changes made to one phone does not break the other phone.

    Should we begin calling it the Nexus Slide? =)

    Update 4/25/2012:
    This March 6th attempt to ship an Epic 4G to Spain seems to have disappeared in the mail.  We have now shipped another Epic, this time USPS Express Mail International with tracking and insurance.

    Update 5/3/2012:
    KalimAz finally received an Epic 4G in Spain!

    Sunday, March 4, 2012

    Details on the Disappearing SD Media Issue

    As recently noted, we're currently testing a set of fixes for the disappearing SD media/ringtone issue that has been plaguing CM users for quite some time.  First of all, we'd like to thank all the users who submitted details, logs, and dumpstates concerning this problem (and others) and being patient while we investigated the cause.  In particular, if you sent any logs/dumpstates to us, we have read them, and although we haven't been able to personally respond to every report, we still want to say "thanks!" for your instrumental role in helping us figure it out.  So, thanks!

    Second, this bug was a real challenge to fix.  Given the strange, "perfect storm"-like conditions needed for it to manifest, and the fact that it affects every CM device with removable storage, we feel it's worth a solid write-up.  So, here we go.

    Symptoms

    Users have been reporting at least since our CM9 alpha1 release that ringtones, notifications, and other media installed to the SD card occasionally "disappear".  In each instance, the actual SD card files remain intact, but they're lost from the media database that's used by Android for selecting ringtones, notifications, Gallery media, etc.  Usually the problem would be resolved in the short term by rebooting or manually forcing a media scan via Dev Tools.  However, the problem would inevitably come back for most users in a couple of days.  Meanwhile, other users (including myself) experienced lost SD media not even once.

    Background

    An Android provider application, MediaProvider, is responsible for maintaining databases that provide details on all media stored on the device.  There are separate databases for media on "internal" and "external" (e.g., SD card) storage volumes.

    When a device is first booted, the MediaScannerService runs to create (if it doesn't already exist) and populate the "internal" media database by scanning every file in /system/media/.  The MediaScannerService also runs every time an SD card is mounted on the device where it creates and populates the "external" media database by scanning every file in /mnt/sdcard/ (except for directories containing a .nomedia file).

    There's three other important features to note about MediaProvider:
    1. While a "card-wide" scan is performed on boot and insertion of SD media, individual applications can register new media files in the media database, at any time, by calling the MediaScannerConnection::scanFile method.
    2. MediaProvider is designed to support multiple "external" media databases on devices that feature removable storage.  Specifically, it's designed to maintain the three most recent external databases, one for each of the three most-recently inserted SD cards, so that one could cycle through a few different SD cards on their device without having to rebuild the database each time.
    3. MediaProvider runs in its own process, "android.process.media", which like any non-critical process, is subject to being killed by ActivityManager so that its resources may be reclaimed.

    Bug #1: Google+'s Repeated Registration of "hangout_ringtone.m4a"

    It turns out that the trigger for SD media disappearance, in every case we've observed, is actually due to the "odd behavior" of a proprietary third-party application, Google+.  In particular, Google+ installs a media file, "hangout_ringtone.m4a", to /mnt/sdcard/Notifications/ on every (or at least, most) device boot-ups.  Its procedure is to first copy "hangout_ringtone.m4a" from its internal resources to /mnt/sdcard/Notifications/, then call the MediaScannerConnection::scanFile method on the new file to register it in the media database.

    We're not exactly certain why Google+ does this on each boot-up, but we do know that often when the copy routine is run, Vold is still performing a file-system consistency check on the SD card.  So the SD card has yet to mount, and thus, the "hangout_ringtone.m4a" appears to not be already installed.

    It's probably a bug that Google+ doesn't check Environment::getExternalStorageState to see if external media is mounted yet before performing the file copy.  However, it's definitely a bug that on receipt on a FileNotFoundException during the copy, that Google+ calls MediaScannerConnection::scanFile on the (non-existent) file anyways.

    Now, this kind of bug would normally be harmless in the absence of additional bugs within Android itself.  Certainly the operating system should be able to tolerate applications doing "odd", but otherwise harmless activities like attempting to media scan a non-existent file.  However, this bug is still notable in understanding the underlying problem as it serves as a trigger condition for underlying OS bugs that we would otherwise not observe in the absence of this program.  In particular, CM users with Google+ installed appear to have experience disappearing SD media rather frequently, whereas users without Google+ experienced it never.  Even among us developers and testers, it was roughly a 50/50 split of who experienced the problem and who didn't, leaving us a bit baffled.

    Bug #2: Media Scanning of (Non-Existent) Files, When the SD Card isn't Mounted, Has the Side Effect of Creating a Second External Media Database

    A private method, MediaProvider::attachVolume is called anytime a media scan is performed, in order to "attach" to either the internal, or an external media database, creating it if necessary, prior to performing the scan.  When attachVolume is called on behalf of an application invoking the MediaScannerConnection::scanFile routine, there's no explicit check if external media is mounted before attaching to the external media database.  For the most part, this isn't problematic itself as the database is eventually created and written to storage once the media is mounted.

    However, on devices featuring removable storage, the attachVolume method selects a unique database name, "external-XXXXXXXX.db" where "XXXXXXXX" is the hexadecimal representation of the SD card's FAT volume ID, e.g., "external-6997f084.db".  The method responsible for looking up the volume ID, FileUtils::getFatVolumeId, returns the ID as an integer quality, and may return "-1" in the event of an error.  Unfortunately, attachVolume doesn't check for this error condition, and thus, if the method is called when an SD card is not (yet) mounted, it will attach to (and later create if necessary) the database "external-ffffffff.db", where "ffffffff" is the 32-bit hexadecimal representation of "-1".

    (An aside: It's not clear me that 0xffffffff isn't a valid FAT volume ID that could be present on an SD card, particularly one that's not formatted by Android.  However, Android implicitly treats it as an invalid ID by reserving that value for the condition of FileUtils.getFatVolumeId's returning error.)

    Now, if the above problem were the only media database-related bug in Android, it wouldn't be a particularly big issue.  The "wrong" database is only created when an SD card isn't mounted, and the media scanner scans the entire contents of the SD card on media mount events.  Furthermore, MediaProvider is designed to maintain the three most-recently used databases, so even if the "right" database were to be attached on a subsequent reboot (or other) event, it would only be slightly stale and the mount-time SD card media scan would bring it up to date.

    Bug #3: The Media Database Garbage Collector is, Well, Broken

    Despite the fact that MediaProvider is supposed to preserve the three most-recently used external media databases, and the fact that Bug #2 erroneously creates only two such databases with a single SD card, we observed that these external databases were being created fresh on attach events, implying that the older of the two databases was constantly being deleted.

    The method responsible for deleting old databases, MediaProvider.DatabaseHelper::onCreate, implements an LRU (least-recently-used) garbage collection policy.  The first thing it does is call android.app.Context::databaseList to find the list of MediaProvider's databases.  The databaseList method, as implemented by android.app.ContextImpl::databaseList, literally returns the list of files in /data/data/com.android.providers.media/databases/.  Next, the garbage collector checks the list for databases last modified over two months ago and deletes them.  Finally, it checks the remaining external databases, preserves the three most-recently used (including the one being attached) and deletes the rest.

    Unfortunately, the directory /data/data/com.android.providers.media/databases/ contains SQLite temporary files (e.g., ".db-shm" and ".db-wal" files) in addition to the SQLite databases themselves.  These files are returned by the ContextImpl::databaseList method, and are considered as distinct "databases" subject to the garbage collector's deletion policies.

    It turns out this breaks the garbage collector in at least two ways.  The first is that the garbage collector may delete one or more of these temporary files, including those associated with the actively-in-use "internal" database.  The other is that these extra files inflate the total number of databases, and so external databases end up being deleted well before the three-database limit is reached.  In short, the garbage collector code works only, and coincidentally, in the single circumstance where exactly one external database exists alongside the internal database.  In the presence of two or more external databases, which we have due to bug #2, the older of the two is inevitably deleted.

    (Aside #2: Although we consider bug #3 to exist within MediaProvider's garbage collector, the "actual" bug may lie in the ContextImpl::databaseList and ContextImpl::deleteDatabase methods, depending on one's perspective.  It's quite reasonable to assume that these methods should return only actual databases, hiding their associated temporary files and handling their deletion internally.  If these methods, which are part of Android's public API and used by third-party applications, were to be modified in this fashion, then the MediaProvider database garbage collector would be correct as is.)

    The "Perfect Storm" Condition

    Even with the above two OS bugs, if MediaProvider were to only attach once to an external database for the duration that SD media is mounted, these problems wouldn't be observed by users.  That's because, regardless of which database is attached to, and whether it's up-to-date, stale, or brand new, the mount-time SD card media scan would still bring it up to date.

    For SD media to disappear, MediaProvider has to attach to external databases at least twice during the same mount period, which as it turns out, does happen.  The exact sequence of events that results in the disappearing of SD media is as follows:
    • Shortly after boot, a third-party application (e.g., Google+) issues a MediaScannerConnection::scanFile on a (non-existent) SD card media file, prior to the SD card being mounted (bug #1).
    • When the scanFile call is issued, MediaProvider attaches to the wrong external database, "external-ffffffff.db", due to the missing error check on the FileUtils.getFatVolumeId call (bug #2).
    • Once the SD card is mounted, MediaProvider creates the "external-ffffffff.db" database and populates it by scanning every file in /mnt/sdcard/.
    • When creating the new, wrong, "external-ffffffff.db" database, the garbage collector deletes the old, right, "external-XXXXXXXX.db" database (bug #3).
    • Some time later, the MediaProvider process is killed by ActivityManager so that its resources may be reclaimed ("perfect storm" condition).
    • Later yet, a new MediaProvider process is spawned to handle a media database query for an incoming call, notification, etc., at which point SD media disappears.
    Now, in the last event, since the SD card is already mounted, the new MediaProvider instance attaches to the right, previously-deleted external database "external-XXXXXXXX.db" and creates it.  Since this happens outside a media mount event, the media scanner does not run and the database remains empty.  And the database will continue to remain empty while processing subsequent ringtone, notification, and other media events until reboot or a forced media scan.

    So that's the story on disappearing SD media, in a nutshell. =)

    Aftermath

    We've created two issues in CM gerrit that patch bugs #2 and #3.  We're currently testing these patches and expect them to be included in the Epic CM9 alpha6 release.  Once they're committed from gerrit, they should make it into following round of CM9 nightlies for other devices.  We're also looking into backporting these patches to CM7 as well.  Hopefully, that'll be the end of it.

    If you appreciate our work, please support the Epic CM team.  Follow us on Google+, Twitter or Facebook.