Coverage-Guided Multi-entry Testing for Android Applications

doi:10.3778/j.issn.1673-9418.2407031

Abstract

Abstract: Improving the quality of Android applications is a significant challenge in the mobile device domain. Activity launch is one of the core functions of an application, and its correctness and reliability will directly affect the user experience and application quality. Recent research indicates that constructing intent test cases can effectively detect issues related to activity launching tests. However, existing solutions still face limitations regarding exploration efficiency and discovering deep crashes. To address these issues, this paper proposes a novel solution called CoMuBot. Firstly, this paper analyzes path constraints to generate appropriate attribute values for intents and combines intent fuzzing techniques to construct fuzzed activity launch contexts, thereby enhancing path coverage and the diversity of test cases. Subsequently, this paper designs a dynamic activity recommendation algorithm and combines it with the state-of-the-art exploration tool APE to achieve an adaptive multi-entry exploration strategy. Finally, incorporating the coverage-guided grey-box fuzzing idea, this paper implements an efficient coverage-guided multi-entry testing method. Experimental results show that the proposed method outperforms the tool Fax regarding exploration efficiency and activity launch accuracy, with a 25.63% increase in method coverage and a high accuracy rate of 96.52% in successfully launching 62.23% of activities. Meanwhile, CoMuBot demonstrates certain advantages over Fax in detecting unique crashes, effectively enhancing the stability and reliability of the application.

Key words: automated graphical user interface testing, inter-component communication（ICC）, fuzzing, symbolic execution, mobile applications

摘要： 提高Android应用程序的质量是移动设备领域所面临的重要挑战。活动启动作为应用程序的核心功能之一，其正确性和可靠性将直接影响用户体验和应用质量。近期的研究表明，构建意图测试用例可有效检测与活动启动测试相关的问题，但现有方案在探索效率和发现深层崩溃方面仍存在限制。针对上述问题，提出一个新颖的解决方案CoMuBot。分析路径约束为意图生成合适的属性值，并结合意图模糊测试技术构建模糊的活动启动上下文，从而提高路径覆盖率和测试用例的多样性。设计动态的活动推荐算法及结合最先进的探索工具APE，实现了一个自适应的多入口探索策略。引入覆盖率引导的灰盒模糊测试思想，实现了一个高效的覆盖导向的多入口测试方法。实验结果表明，该方法在探索效率和活动启动准确性方面均优于工具Fax，其中方法覆盖率提升了25.63%，并以高达96.52%的准确率成功启动了62.23%的活动。同时，在独特崩溃检测方面，CoMuBot相比于Fax和DALT展现出了一定的优势，有效提高了应用程序的稳定性和可靠性。

关键词: 自动化图形用户界面测试, 组件间通信（ICC）, 模糊测试, 符号执行, 移动应用程序

FU Jie, GUO Fan. Coverage-Guided Multi-entry Testing for Android Applications[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(6): 1666-1680.

傅洁, 郭帆. 覆盖导向的Android程序多入口测试方法[J]. 计算机科学与探索, 2025, 19(6): 1666-1680.

References

[1] Market Data. Top apps and games[EB/OL]. [2024-05-10]. https://www.data.ai/cn/insights/market-data/q1-2023-top-apps-and-games/.
[2] ZHENG W, LIN L D, CHEN X, et al. ISTA: automatic test case generation and optimization for intelligent systems based on coverage analysis[C]//Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering. Piscataway: IEEE, 2023: 758-762.
[3] WANG W Y, LI D F, YANG W, et al. An empirical study of Android test generation tools in industrial cases[C]//Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. New York: ACM, 2018: 738-748.
[4] 丁蕊, 董红斌, 张岩, 等. 基于关键点路径的快速测试用例自动生成方法[J]. 软件学报, 2016, 27(4): 814-827.
DING R, DONG H B, ZHANG Y, et al. Fast automatic generation method for software testing data based on key-point path[J]. Journal of Software, 2016, 27(4): 814-827.
[5] CHOUDHARY S R, GORLA A, ORSO A. Automated test input generation for Android: are we there yet?(E)[C]//Proceedings of the 2015 30th IEEE/ACM International Conference on Automated Software Engineering. Piscataway: IEEE, 2015: 429-440.
[6] 张文, 陈锦富, 蔡赛华, 等. 一种聚类分析驱动种子调度的模糊测试方法[J]. 软件学报, 2024, 35(7): 3141-3161.
ZHANG W, CHEN J F, CAI S H, et al. Fuzzing approach of clustering analysis-driven in seed scheduling[J]. Journal of Software, 2024, 35(7): 3141-3161.
[7] 钱忠胜, 宋涛. 面向关键字流图的相似程序间测试用例的重用[J]. 软件学报, 2021, 32(9): 2691-2712.
QIAN Z S, SONG T. Reuse of test cases between similar programs based on keyword flow graph[J]. Journal of Software, 2021, 32(9): 2691-2712.
[8] FENG S D, CHEN C Y. GIFdroid: automated replay of visual bug reports for Android apps[C]//Proceedings of the 2022 IEEE/ACM 44th International Conference on Software Engineering. Piscataway: IEEE, 2022: 1045-1057.
[9] ZHAO Y, YU T T, SU T, et al. ReCDroid: automatically reproducing Android application crashes from bug reports[C]//Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering. Piscataway: IEEE, 2019: 128-139.
[10] 曹羽中, 吴国全, 陈伟, 等. 一种基于录制/重放的Android应用众包测试方法[J]. 软件学报, 2020, 31(8): 2508-2529.
CAO Y Z, WU G Q, CHEN W, et al. Crowdsourcing test method for Android applications based on recording/replay[J]. Journal of Software, 2020, 31(8): 2508-2529.
[11] SASNAUSKAS R, REGEHR J. Intent fuzzer: crafting intents of death[C]//Proceedings of the 2014 Joint International Workshop on Dynamic Analysis and Software and System Performance Testing, Debugging, and Analytics. New York: ACM, 2014: 1-5.
[12] HAY R, TRIPP O, PISTOIA M. Dynamic detection of inter-application communication vulnerabilities in Android[C]//Proceedings of the 2015 International Symposium on Software Testing and Analysis. New York: ACM, 2015: 118-128.
[13] RASTHOFER S, ARZT S, TRILLER S, et al. Making Malory behave maliciously: targeted fuzzing of Android execution environments[C]//Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering. Piscataway: IEEE, 2017: 300-311.
[14] YAN J W, LIU H, PAN L J, et al. Multiple-entry testing of Android applications by constructing activity launching contexts[C]//Proceedings of the 2020 IEEE/ACM 42nd International Conference on Software Engineering. Piscataway: IEEE, 2020: 457-468.
[15] LIU A, GUO C K, DONG N P, et al. DALT: deep activity launching test via intent-constraint extraction[C]//Proceedings of the 2022 IEEE 33rd International Symposium on Software Reliability Engineering. Piscataway: IEEE, 2022: 482-493.
[16] FU J. The access address for CoMuBot[EB/OL]. [2024-05-10]. https://github.com/InterestingApple/CoMuBot.
[17] LIU J R, WU T Y, DENG X, et al. InsDal: a safe and extensible instrumentation tool on Dalvik byte-code for Android applications[C]//Proceedings of the 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering. Piscataway: IEEE, 2017: 502-506.
[18] GU T X, SUN C N, MA X X, et al. Practical GUI testing of Android applications via model abstraction and refinement[C]//Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering. Piscataway: IEEE, 2019: 269-280.
[19] NIELSON F, NIELSON H R, HANKIN C. Principles of program analysis[M]. Berlin, Heidelberg: Springer, 1999.
[20] MOURA L D, BJ?RNER N. Z3: an efficient smt solver[C]//Proceedings of the 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Berlin, Heidelberg: Springer, 2008: 337-340.
[21] LIU C L. ProMal: precise window transition graphs for Android via synergy of program analysis and machine learning[C]//Proceedings of the 2021 IEEE/ACM 43rd International Conference on Software Engineering. Piscataway: IEEE, 2021: 144-146.
[22] CHEN S, FAN L L, CHEN C Y, et al. Automatically distilling storyboard with rich features for Android apps[J]. IEEE Transactions on Software Engineering, 2023, 49(2): 667-683.
[23] YANG S Q, WU H W, ZHANG H L, et al. Static window transition graphs for android[J].?Automated Software Engineering,?2018, 25: 833-873.
[24] The access address for APKTool[EB/OL]. [2024-05-12]. https:// github.com/iBotPeaches/Apktool.
[25] SOOT-OSS. The access address for Soot[EB/OL]. [2024-05-12]. https://github.com/soot-oss/soot.
[26] The access address for Jadx[EB/OL]. [2024-05-12]. https://github.com/skylot/jadx.
[27] Android Developers. Ui/application exerciser monkey[EB/OL]. [2024-05-12]. https://developer.android.com/studio/test/other-testing-tools/monkey.
[28] YAZDANIBANAFSHEDARAGH F, MALEK S. Deep GUI: black-box GUI input generation with deep learning[C]//Proceedings of the 2021 36th IEEE/ACM International Conference on Automated Software Engineering. Piscataway: IEEE, 2021: 905-916.
[29] WANG J, JIANG Y Y, XU C, et al. ComboDroid: generating high-quality test inputs for Android apps via use case combinations[C]//Proceedings of the 2020 IEEE/ACM 42nd International Conference on Software Engineering. Piscataway: IEEE, 2020: 469-480.
[30] SU T, MENG G Z, CHEN Y T, et al. Guided, stochastic model-based GUI testing of Android apps[C]//Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. New York: ACM, 2017: 245-256.
[31] KURIAN E, BRIOLA D, BRAIONE P, et al. Automatically generating test cases for safety-critical software via symbolic execution[J]. Journal of Systems and Software, 2023, 199: 111629.
[32] AGGARWAL A, LOHIA P, NAGAR S, et al. Black box fairness testing of machine learning models[C]//Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. New York: ACM, 2019: 625-635.
[33] MAO K, HARMAN M, JIA Y. Sapienz: multi-objective automated testing for Android applications[C]//Proceedings of the 25th International Symposium on Software Testing and Analysis. New York: ACM, 2016: 94-105.
[34] LV Z W, PENG C, ZHANG Z, et al. Fastbot2: reusable automated model-based GUI testing for Android enhanced by reinforcement learning[C]//Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering. New York: ACM, 2022: 1-5.
[35] ROMDHANA A, MERLO A, CECCATO M, et al. Deep reinforcement learning for black-box testing of Android apps[J]. ACM Transactions on Software Engineering and Methodology, 2022, 31(4): 1-29.
[36] PAN M X, HUANG A, WANG G X, et al. Reinforcement learning based curiosity-driven testing of Android applications[C]//Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. New York: ACM, 2020: 153-164.
[37] 张少坤, 李元春, 雷瀚文, 等. 基于多模态表征的移动应用GUI模糊测试框架[J]. 软件学报, 2024, 35(7): 3162-3179.
ZHANG S K, LI Y C, LEI H W, et al. GUI fuzzing framework for mobile apps based on multi-modal representation[J]. Journal of Software, 2024, 35(7): 3162-3179.
[38] BURNS J. Null intent fuzzer[EB/OL]. [2024-05-12]. https://www.isecpartners.com/tools/mobile-security/intent-fuzzer.aspx.
[39] ARZT S, RASTHOFER S, FRITZ C, et al. FlowDroid: precise context, flow, field, object-sensitive and lifecycle-aware taint analysis for Android apps[C]//Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation. New York: ACM, 2014: 259-269.
[40] CAO S C, HE B, SUN X B, et al. ODDFuzz: discovering Java deserialization vulnerabilities via structure-aware directed greybox fuzzing[C]//Proceedings of the 2023 IEEE Symposium on Security and Privacy. Piscataway: IEEE, 2023: 2726-2743.
[41] DU Z J, LI Y K, LIU Y, et al. Windranger: a directed greybox fuzzer driven by deviation basic blocks[C]//Proceedings of the 2022 IEEE/ACM 44th International Conference on Software Engineering. Piscataway: IEEE, 2022: 2440-2451.
[42] LEE G, SHIM W, LEE B. Constraint-guided directed greybox fuzzing[C]//Proceedings of the 30th USENIX Security Symposium. Berkeley: USENIX Association, 2021: 3559-3576.
[43] WANG Y H, JIA X K, LIU Y W, et al. Not all coverage measurements are equal: fuzzing by coverage accounting for input prioritization[C]//Proceedings of the 2020 Network and Distributed System Security Symposium, 2020.
[44] WANG J H, SONG C Y, YIN H. Reinforcement learning-based hierarchical seed scheduling for greybox fuzzing[C]//Proceedings of the 2021 Network and Distributed System Security Symposium, 2021.
[45] YUE T, WANG P F, TANG Y, et al. Ecofuzz: adaptive energy-saving greybox fuzzing as a variant of the adversarial multi-armed bandit[C]//Proceedings of the 29th USENIX Security Symposium. Berkeley: USENIX Association, 2020: 2307-2324.
[46] LYU C, JI S L, ZHANG X H, et al. EMS: history-driven mutation for coverage-based fuzzing[C]//Proceedings of the 29th Annual Network and Distributed System Security Symposium, 2022.