Showing posts with label android. Show all posts
Showing posts with label android. Show all posts

Wednesday, April 11, 2012

Lesson Learned form Developing Locadz SDK

這是我 4/25 號要在 GTUG 發表的東西,先把講稿寫在 blog 上。

Introduction


市面上大部份講 Android 的書,多數都是在講 API 要怎麼樣使用,但是,許多的書都沒有提到一個重要的問題,就是什麼是 UI Thread (有時被稱做Main Thread) ,為什麼要有 UI Thread 以及為什麼不可以在 UI Thread 中執行耗時間的運算。

在本文中,我講談一談我們在開發Locadz SDK所使用到的一些技巧。

UI Thread


UI Thread是Android(or other GUI library)用來處理 Event 的 thread ,這些 Event 可能是使用者處發的,如Touch Event,或者是其它程式或者是系統底層的事件,如 Intent 或 Location Updates

在 UI Thread 處理完一個事件後, UI Thread 會重畫整個 Application ,而前面這句話解釋了處理 ANR 的兩個原則
  • UI事件的處理要越快越好,在處理完前,UI不會有反應
  • UI的更新,一定要發生在UI Thread,要不然,要等到下次UI Event被處發時才會一併被處理



Lesson I: Use WeakReference to Avoid Strong Reference and Memory Leak


有許多的文章談到如何用 AsyncTask ResultReceiver 來把需要大量運算的程式,放在另一個 Thread 來做處理。

然而,這些範例都有著一個常備忽略的問題,那就是,這些 AsyncTask or ResultReceiver 常會把前景的Activity包進來,如果說你的程式只有一個 Activity 的話,這或許不是什麼問題,但是若是你的 App 有多個畫面的話,這些在背景執行的程式可能就會讓你的程式有 memory leak 的問題;如某家廣告商的 SDK 就有此問題。

一個可能的發生狀況是,Activity A透過 AsyncTask 去遠端下載一個圖片來顯示在Activity A之上,但因為某些因素(如忘了設 connection timeout),這個下載的程緒卡住了,既使 User 以從Activity A切換到Activity B之上,這個Activity A還是不會被 GC 回收掉。


要避開因為有個 Strong Reference 造成 Activity 無法被回收的問題,我們在把有可能被回收的物件傳到另一個 Thread 中被延後執行時,必需用 WeakReference包住這個物件;如此一來,當Garbage Collector碰到一個已經不在前景的 Activity 時,Garbage Collector會把這物件處理掉,如此一來,就不會有 memory leak 的問題。

/**
 *  AsyncTask to Load Image
 */
public class DownloadImagesTask extends AsyncTask<Uri, Void, Bitmap> {

  WeakReference<imageview> imageViewWeakReference = null;

  public DownloadImagesTask(ImageView imageView) {
      imageViewWeakReference = new WeakReference<Imageview>(imageView);
  }

  @Override
  protected Bitmap doInBackground(Uri... uri) {
      return downloadImage(uri);
  }

  @Override
  protected void onPostExecute(Bitmap result) {
    ImageView imageView = imageViewWeakReference.get();
    if (imageView != null) {
      imageView.setImageBitmap(result);
    }
  }

  private Bitmap downloadImage(Uri url) {
     ...
  }



Lesson II: Use IntentService to Run Business Logic


AsyncTask是 Android 最常被用來處理複雜運算時用的工具,透過AsyncTask,我們可以在背景處裡一些複雜的運算,再把結果放回前景之上。

但據我的經驗,使用 AsyncTask 同時間來擔任 MVC 中的 View & Controller 的工作,最後往往是把程式碼弄成一團麵線。因此,在開發 Locadz SDK 時,我們把一些跟 UI 無關的運算,都切出來變成 IntentService或者是非Inner class的AsyncTask,把所有的運算邏輯從 Activity 中切出來,增加重用的可能。

然後運算的結果,再透過 getHandler().post(...) 更新到 UI 之上.

/** Service that retrieve the ad unit allocations from external source and cache locally in SharedPreference. */
public class AdUnitAllocationService extends IntentService {

    private static final int CACHE_EXPIRATION_PERIOD = 30 * 60 * 1000; // 30 minutes.

    private final static String PREFS_STRING_TIMESTAMP = "timestamp";
    private final static String PREFS_STRING_CONFIG = "config";

    // response code for possible result.
    public static final int RESULT_OK = 1;
    
    public AdUnitAllocationService() {
        super(AdUnitAllocationService.class.getCanonicalName());
    }

    @Override
    protected void onHandleIntent(Intent intent) {

        AdUnitContext adUnitContext = 
           (AdUnitContext) intent.getParcelableExtra(IntentConstants.EXTRA_ADUNIT_CONTEXT);

        AdUnitAllocation adUnitAllocation = getAdUnitAllocation(adUnitContext);

        if (adUnitAllocation != null) {
            Ration ration = getRandomRation(adUnitAllocation.getRations());

            // send response through ResultReceiver.
            ResultReceiver receiver = intent.getParcelableExtra(IntentConstants.EXTRA_RECEIVER);

            Bundle resultData = new Bundle();
            resultData.putString(IntentConstants.EXTRA_ADUNIT_ID, adUnitContext.getAdUnitId());
            resultData.putSerializable(IntentConstants.EXTRA_RATION, ration);
            resultData.putSerializable(IntentConstants.EXTRA_EXTRA,
                                       adUnitAllocation.getExtra());

            receiver.send(RESULT_OK, resultData);
        }
    }

    /**
     * Select a random ration form the provided rations.
     * @param rations   the candidates.
     * @return a random ration from the candidates.
     */
    private Ration getRandomRation(List<Ration> rations) {
        //...
    }

    /**
     * Get the allocation configuration for the adunit.
     * @param adUnitContext the context of the adunit.
     * @return the allocation configuration for the adunit.
     */
    AdUnitAllocation getAdUnitAllocation(AdUnitContext adUnitContext) {
        //...
    }
}



Lesson III: Use Disk Cache instead of (Main) Memory Cache



底下的圖表,是Jeff Dean發表的,在談的是讀取資料的的效率,我們把這幾個數字先記起來,再加一個代表UI設計時人體覺得是即時反應的反應時間上限 100 ms。然後我們再來談 Android UI 的設計。



大家可以看到 Main Memory Reference(0.001ms) 與 Disk Seek(10ms) Disk Read(30ms) 的重大差距,然而,後者的數字在 Mobile Phone 上就不是這樣了。在 Mobile Phone 上,傳統的硬碟扮演的角色,被NAND Flash Memory, External SD Card所取代了。在存取效率上 NAND Flash Memory 雖然不比 RAM 快,但是,也仍是 seek time ~1ms 的狠角色。

這 1ms 的負擔,雖比 0.001ms 的負擔高上百倍,以上,但是在 100ms 這UI 反應需求上,卻又變得很渺小了。

因此,在這邊,我會建議大家,若是有 cache 的需求時,直接往 Internal Storage 塞吧,不要放在Main memory上,或用個SoftReferenceMap包著。



Lesson IV: Make All Public Method Async to Avoid UI Update Issue


在上面第一個範例中有個錯誤,那就是DownloadImagesTask.onPostExecute()會在呼叫DownloadImagesTask.execute(...)的那個 Thread 上執行,如果說,很不幸的,這個 DownloadImagesTask 並不是從 UI Thread 上來呼叫的話,那麼,imageView.setImageBitmap(result)便有可能不會即時更新到UI之上。

如果你的開發環境會有這種問題,在包在層層呼叫後,無法確保 Method 是否是在 UI Thread 上執行;那麼我會建議你把會更新 UI 的 Method ,標成 protected ,然後開放一個 public async method 出來,範例如下:

/**
     * Remove old ad views and push the new one.
     *
     * @param subView the adview to push.
     */
    protected void pushSubView(ViewGroup subView) {
        //....
    }

    /**
     *  submit a push view request to Android's handler. This will remove
     *  old ad view and push a new one to this layout asynchronously.
     *
     * @param subView   the adview to push.
     */
    public void submitPushSubViewRequest(ViewGroup subView) {
        Log.d(LOG_TAG, String.format("Scheduled pushSubView(%s)", subView));
        getHandler().post(new ViewAdRunnable(this, subView));
    }


    /**
     * Runnable runs on the Main Thread that pushes an AdView to the layout.
     */
    private static final class ViewAdRunnable implements Runnable {

        private final WeakReference<Adunitlayout> locadzLayoutWeakReference;

        private ViewGroup subView;

        public ViewAdRunnable(AdUnitLayout layout, ViewGroup subView) {
            locadzLayoutWeakReference = new WeakReference<Adunitlayout>(layout);
            this.subView = subView;
        }

        @Override
        public void run() {
            AdUnitLayout locadzLayout = locadzLayoutWeakReference.get();
            if (locadzLayout != null) {
                locadzLayout.pushSubView(subView);
            }
        }
    }


Thursday, December 8, 2011

Lucene On Android

嘗試性的把 Lucene 放到 Android 上面來跑,結果不是太理想,但仍是一些心得分享出來,省去後人嘗試的時間。

Lucene 要跑在 Android 上,第一個碰上的問題是,如何把 index files 傳到手機上去,在 Lucene 中,對 index 的讀取,是以目錄為單位的,所以說,無法把 index files 放在 apk 中直接讀取,一定要存放在 device or external storage 上,才能夠使用;或者是自己弄個虛擬目錄出來,不過,這會耗用過多的計憶體空間。

我是選用把 index 放在 'src/res/raw' 底下,讓他變成 apk 的一部份,省去在網路上找個空間來放置 index 的問題,當要更新 index 時,就重編個 apk 叫使用者更新就好。

在放 lucene index 時,如果你想用 compound format 的話,可以用底下的指令,把多個 index files 包裹成單一檔案 .cfs

// run REPL with 'scala -cp luke-3.4.0_1-all.jar'

import org.apache.lucene.store.FSDirectory
import org.apache.lucene.index._
import org.apache.lucene.analysis.standard._

val source = FSDirectory.open(new java.io.File("source"))
val dest = FSDirectory.open(new java.io.File("dest"))

// open source index
val reader = IndexReader.open(source)

// create writer for compound index.
val analyzer = new StandardAnalyzer(org.apache.lucene.util.Version.LUCENE_34)
val writer = new org.apache.lucene.index.IndexWriter(dir, analyzer, IndexWriter.MaxFieldLength.UNLIMITED)

// force writer always use compound index format.
writer.getMergePolicy.asInstanceOf[LogByteSizeMergePolicy].setNoCFSRatio(1.0)


// add source index to dest index.
writer.addIndexes(reader)
writer.optimized
writer.close

reader.close

接著,是把產生的 segment, segments_1, _0.cfs 拷到 src/res/raw 底下,讓這些檔案變成 .apk 的一部份。


接著,是要在第一次執行時,把這些 index 從 apk 中覆製到 SD 卡上或是機子上,這邊,我寫了個小工具來做這件事

import android.content.Context
import android.os.Environment
import android.util.Log

import com.bluetangstudio.android.disastermap.TaipeiDisasterApp.LogTag

import org.apache.commons.io.FileUtils
import org.apache.lucene.store.{FSDirectory, Directory}

import scala.collection.JavaConversions._
import java.io.File

/**
 *  Helper class that search for lucene index directories on the device. The search order is
 *  external storage first then local storage. If lucene index does not exist on device, this
 *  class will copy the index from the apk to the device storage.
 *
 * @param context  the application context
 * @param path     the root folder name of the index directory to use and to look for.
 * @param source   the source of index resource to copy if index does not exist on the device.
 *                 format: Seq[(filename, resourceId)]
 */
case class LuceneOpenHelper(context: Context, path: String, source: Seq[Tuple2[String, Int]]) {

    /**
     * create or open an Directory.
     */
    def open(): Option[Directory] = {
        val candidates = Seq(externalFolder, internalFolder).flatten

        // find the folder with index in it.
        val folder = candidates.filter(f => f.exists() && f.list().length > 0).headOption
        val withIndex = folder.orElse(
            candidates.find(
                f => {
                    // ensure folder is available.
                    f.exists() || f.mkdirs() match {
                        // folder is not accessible
                        case false => false

                        case _ => {
                            Log.d(LogTag, "Duplicating index from apk to %s...".format(f))
                            source.foreach(
                                s => {
                                    val is = context.getResources.openRawResource(s._2)
                                    try {
                                        FileUtils.copyInputStreamToFile(is, new File(f, s._1))
                                    } finally {
                                        is.close()
                                    }
                                }
                            )
                            true
                        }
                    }
                }
            )
        )

        return withIndex.map(FSDirectory.open(_))
    }

    private def externalFolder: Option[File] = {
        Environment.getExternalStorageState match {
            case Environment.MEDIA_MOUNTED => Option(context.getExternalFilesDir(path))
            case _ => None
        }
    }

    private def internalFolder: Option[File] = {
        return Option(new File(context.getFilesDir, path))
    }

}

最後,是在 *App 上加上這段

object MyApp {
    private val INDEX_DIRECTORY = "idx"

    private val INDEX_FILES = Seq(
        ("_0.cfs", R.raw.idx_0), 
        ("segments", R.raw.segments), 
        ("segments_1", R.raw.segments_1)
    )
}
class MyApp extends android.app.Application {

    import MyApp._

    private var _luceneSearcher: Option[IndexSearcher] = None

    def luceneSearcher: Option[IndexSearcher] = {
        if (_luceneSearcher.isEmpty) {
            Log.d(LogTag, "Initializing new IndexSearcher...")
            _luceneSearcher = LuceneOpenHelper(this, INDEX_DIRECTORY, INDEX_FILES).open().map(new IndexSearcher(_))
        }
        _luceneSearcher
    }
   
    override def onLowMemory() {
        _luceneSearcher.foreach(s => s.close())
        _luceneSearcher = None
    }
}

這樣一來,就能在 Android 上面跑 lucene-core 了.

Tuesday, November 22, 2011

Source code of my Scala on Android Project.

把前面幾回講的用Scala寫的Android程式,放上 bitbucket 了,有興趣的可以自己去下載來看。

https://bitbucket.org/bluetang/android-taipei-disaster

Wednesday, November 16, 2011

My experience with Scala on Android

用Scala開發Android,是個滿有趣的經驗,大致上來因為我切入的時間點較晚,所以大部份的問題已經被前人所解決,就用 https://github.com/jberkel/android-plugin 把專案用 sbt 開好後,就可以開始寫 android.

我用的 IDE 是 intellij 11 EAP,用 sbt-idea 把 idea project 設好後,把預設的 asset pa th 從 .idea_module 改成 src/main ,就可以開始開發了。


在開發上,除了前一回碰上的proguard問題外,我還碰上另一個比較嚴重的問題-不能在Scala裡寫 AsyncTask

這問題跟SI-3622 SI-3494有關,看來是在 Scala 2.8解掉的問題,2.9又跑回來了,我這邊看到的狀況是

override protected def doInBackground(params: Params*): Result

會被Scala compiler專換成
public Result doInBackground(Seq params)
    public Result doInBackground(Params[] params)


而底下的code,則是scala compiler會吐出 overides nothing.
override protected def doInBackground(params: Array[Params]): Result

不管怎樣,都跟Android要求的protected Result doInBackground(Params[] params)不同,所以在runtime時會跑出NoSuchMethodError.

解決方法是在 java 裡寫個 bridge interface ,幫 scala compiler 搞不定的東西,在這個 interface 裡定意好

public abstract class SAsyncTask<Params, Progress, Result> extends AsyncTask<Params, Progress, Result> {

    protected abstract Result doInBackground(Seq<Params> params);

    @Override
    protected Result doInBackground(Params... paramses) {
        return doInBackground(JavaConversions.asScalaBuffer(Arrays.asList(paramses)));
    }
}

Tuesday, November 15, 2011

Data Modeling With Jackson Json and Scala - Proguard

關於Proguard,官方的網頁是這麼自述的:
ProGuard is a free Java class file shrinker, optimizer, obfuscator, and preverifier. It detects and removes unused classes, fields, methods, and attributes. It optimizes bytecode and removes unused instructions. It renames the remaining classes, fields, and methods using short meaningless names. Finally, it preverifies the processed code for Java 6 or for Java Micro Edition.

在Mobile App開發的時候,多會用proguard把沒有用到的程式碼濾除,並把變數名稱跟函式名稱用更精簡的字串來取代,這樣編出來的程式會更小,以我用Scala開發的Android程式,使用的的函式庫大小超過10MB,但是用proguard編出來的class.min.jar只有1.7MB,8.xMB的scala-runtime.jar一大票沒用到的功能都被移掉了。

這麼好的功具當然也有他的問題存在,proguard是使用靜態分析的方式,去追縱看有那些程式碼會被執行到,有那些程式碼是不會被碰到的可以被移除的;但由於Jackson是使用Reflection的方式去取得物件的屬性及是用reflection的方式去生成物件,因此,這些行為並不會被proguard偵測到,反而是被認為是dead code而被移除掉。

另外proguard會把變數名稱改寫,這也是跟jackson不相容的地方,當getXxxx被改寫成gY,jackson自然無法知道這是Xxxx的getter,因此若是要在proguard的環境下使用jackson,obfuscator是要被關掉的。

底下這邊,是我在台北積水地圖內,用的proguard設定檔

proguardOption in Android :=
      ("-dontoptimize -dontpreverify -dontobfuscate"  // shrinking only
          :: "-dontskipnonpubliclibraryclassmembers"  // keep Jackson's internal classes
          :: "-dontskipnonpubliclibraryclasses"       // keep Jackson's internal classes
          :: "-keepattributes *Annotation*."          // keep Jackson Json Annotations.
          :: "-keep class org.codehaus.jackson.**"
          :: "-keep class com.bluetangstudio.android.model.**"
          :: "-keep class com.bluetangstudio.searchcloud.client.**"
          :: """-keep class com.bluetangstudio.searchcloud.client.** {
                 (...);
                 public static ** valueOf(...);
             }"""
          :: """-keep class com.bluetangstudio.** {
                 void set*(***);
                 void set*(int, ***);

                 boolean is*();
                 boolean is*(int);

                 *** get*();
                 *** get*(int);
             }"""
          :: """-keep class * implements android.os.Parcelable {
                 public static final android.os.Parcelable$Creator *;
             }"""
          :: """-keepclassmembers class * {
                 ** MODULE$;
             }"""
          :: "-keep public class org.xml.sax.EntityResolver"
          :: "-keep public class scala.Either"
          :: "-keep public class scala.Function1"
          :: "-keep public class scala.Function2"
          :: "-keep public class scala.Tuple2"
          :: "-keep public class scala.collection.Iterable"
          :: "-keep public class scala.PartialFunction"
          :: "-keep public class scala.collection.Seq"
          :: "-keep public class scala.collection.TraversableOnce"
          :: "-keep public class scala.collection.generic.CanBuildFrom"
          :: "-keep public class scala.collection.immutable.Map"
          :: "-keep public class scala.collection.immutable.List"
          :: "-keep public class scala.collection.mutable.StringBuilder"
          :: "-keep public class scala.Predef$$less$colon$less"
          :: "-keep public class scala.math.Numeric"
          :: "-keep public class scala.math.Ordering"
          :: "-keep public class scala.reflect.ClassManifest"
          :: "-keep public class scala.runtime.IntRef"
          :: "-keep public class scala.runtime.BooleanRef"
          :: "-keep public class scala.runtime.AbstractFunction1"
          :: "-keep public class * extends android.app.Activity"
          :: "-keep public class * extends android.app.Application"
          :: "-keep public class * extends android.app.Service"
          :: "-keep public class * extends android.appwidget.AppWidgetProvider"
          :: "-keep public class * extends android.content.BroadcastReceiver"
          :: "-keep public class * extends android.content.BroadcastReceiver"
          :: "-keep public class * extends android.app.backup.BackupAgentHelper"
          :: "-keep public class * extends android.content.ContentProvider"
          :: "-keep public class * extends android.view.View"
          :: "-keep public class * extends android.preference.Preference"
          :: Nil
      ) mkString " "