Data-Driven Release Governance for Embedded Software Systems in Consumer Devices
DOI:
https://doi.org/10.22399/ijcesen.5548Keywords:
Data-driven release governance, Embedded software systems, Telemetry analytics, Machine learning in software engineering, Over-the-air updatesAbstract
Governance of data-driven release has become an important paradigm to handle the increased complexity and size of embedded software systems in consumer devices. The proliferation of networked devices and the growing propensity towards over-the-air update systems has placed a strain on more adaptable, robust and evidence-based software release administration. This review is a critical assessment of the current literature on the notion of data-driven release governance, and its methodologies, tools of analysis and system-level implications in particular. The analysis addresses the wide range of strategies, including supervised and unsupervised machine learning models, analytics in telemetry, probabilistic risk modelling, and optimization via reinforcement learning. The evidence of the use of data-driven techniques to enhance the quality of releases, reduce failures, and enhance system reliability is reported in the literature. Telemetry data are key in enabling continuous monitoring and proactive detection of anomalies, and models of hybrid governance between automated analytics and human control have been found to be effective in safety-critical contexts. Regardless of these developments, there are a number of issues, such as the heterogeneity of data, poor observability, scalability, and the requirement of model interpretability. The barriers that have to be overcome in order to ensure that data-driven methods are successful are organizational and technical since they involve integrating these approaches into embedded system pipelines. Emerging research directions such as federated learning, causal inference, edge analytics and digital twin technologies are some of the potential ways out of these limitations. In general, this review sheds light on the transformative opportunities of data-driven release governance and outlines key gaps and opportunities in future research. The results highlight the need to build scale-out, understandable and context-sensitive solutions to next-generation embedded software systems.
References
[1] Humble, J., & Farley, D. (2010). Continuous delivery: Reliable software releases through build, test, and deployment automation. Addison-Wesley.
[2] Menzies, T., Zimmermann, T., Bird, C., & Nagappan, N. (2013). The promise and perils of mining Git. IEEE Software, 30(1), 22–29.
[3] Mahapatra, S., & Mohapatra, P. (2019). Over-the-air software updates in connected vehicles: A survey. IEEE Communications Surveys & Tutorials, 21(4), 3490–3515.
[4] Lessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008). Benchmarking classification models for software defect prediction: A proposed framework and novel findings. IEEE Transactions on Software Engineering, 34(4), 485–496. DOI: https://doi.org/10.1109/TSE.2008.35
[5] Mao, K., Harman, M., & Jia, Y. (2016). Sapienz: Multi-objective automated testing for Android applications. Proceedings of the 25th International Symposium on Software Testing and Analysis, 94–105. DOI: https://doi.org/10.1145/2931037.2931054
[6] Chen, T. Y., Kuo, F.-C., Merkel, R. G., & Tse, T. H. (2010). Adaptive random testing: The ART of test case diversity. Journal of Systems and Software, 83(1), 60–66. DOI: https://doi.org/10.1016/j.jss.2009.02.022
[7] Xu, L., Xu, C., & Cheung, S. C. (2018). Improving the performance of log-based anomaly detection systems. IEEE Transactions on Reliability, 67(3), 1057–1071.
[8] Barreno, M., Nelson, B., Joseph, A. D., & Tygar, J. D. (2010). The security of machine learning. Machine Learning, 81(2), 121–148. DOI: https://doi.org/10.1007/s10994-010-5188-5
[9] Zaharia, M., Das, T., Li, H., Hunter, T., Shenker, S., & Stoica, I. (2013). Discretized streams: Fault-tolerant streaming computation at scale. Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, 423–438. DOI: https://doi.org/10.1145/2517349.2522737
[10] Hall, T., Beecham, S., Bowes, D., Gray, D., & Counsell, S. (2012). A systematic literature review on fault prediction performance in software engineering. IEEE Transactions on Software Engineering, 38(6), 1276–1304. DOI: https://doi.org/10.1109/TSE.2011.103
[11] Patel, D. G. (2025). Generative AI in DevOps: Enhancing cloud workflow automation. International Journal of Innovative Research in Technology, 12(2), 3732–3742.
[12] Zimmermann, T., & Nagappan, N. (2008). Predicting defects using network analysis on dependency graphs. Proceedings of the 30th International Conference on Software Engineering, 531–540. DOI: https://doi.org/10.1145/1368088.1368161
[13] Menzies, T., Greenwald, J., & Frank, A. (2007). Data mining static code attributes to learn defect predictors. IEEE Transactions on Software Engineering, 33(1), 2–13. DOI: https://doi.org/10.1109/TSE.2007.256941
[14] Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359. DOI: https://doi.org/10.1109/TKDE.2009.191
[15] He, P., Zhu, J., He, S., Li, J., & Lyu, M. R. (2016). An evaluation study on log parsing and its use in log mining. Proceedings of the 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, 654–661. DOI: https://doi.org/10.1109/DSN.2016.66
[16] Pang, G., Shen, C., Cao, L., & van den Hengel, A. (2021). Deep learning for anomaly detection: A review. ACM Computing Surveys, 54(2), Article 38. DOI: https://doi.org/10.1145/3439950
[17] Fenton, N., & Neil, M. (2012). Risk assessment and decision analysis with Bayesian networks. CRC Press. DOI: https://doi.org/10.1201/9780367803018
[18] Weber, P., Medina-Oliva, G., Simon, C., & Iung, B. (2012). Overview on Bayesian networks applications for dependability, risk analysis and maintenance areas. Engineering Applications of Artificial Intelligence, 25(4), 671–682. DOI: https://doi.org/10.1016/j.engappai.2010.06.002
[19] Ståhl, D., & Bosch, J. (2014). Modeling continuous integration practice differences in industry software development. Journal of Systems and Software, 87, 48–59. DOI: https://doi.org/10.1016/j.jss.2013.08.032
[20] Kim, S., Zimmermann, T., Pan, K., & Whitehead, E. (2007). Automatic identification of bug-introducing changes. Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering, 81–90. DOI: https://doi.org/10.1109/ASE.2006.23
[21] Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction (2nd ed.). MIT Press.
[22] Kohavi, R., Tang, D., & Xu, Y. (2020). Trustworthy online controlled experiments: A practical guide to A/B testing. Cambridge University Press. DOI: https://doi.org/10.1017/9781108653985
[23] Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys, 51(5), Article 93. DOI: https://doi.org/10.1145/3236009
[24] Catal, C., & Diri, B. (2009). A systematic review of software fault prediction studies. Expert Systems with Applications, 36(4), 7346–7354. DOI: https://doi.org/10.1016/j.eswa.2008.10.027
[25] Sakurada, M., & Yairi, T. (2014). Anomaly detection using autoencoders with nonlinear dimensionality reduction. Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis, 4–11. DOI: https://doi.org/10.1145/2689746.2689747
[26] Box, G. E. P., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: Forecasting and control (5th ed.). Wiley.
[27] Pearl, J. (2009). Causality: Models, reasoning, and inference (2nd ed.). Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511803161
[28] Bertsekas, D. P. (2019). Reinforcement learning and optimal control. Athena Scientific.
[29] Bosch, J. (2014). Continuous software engineering: An emerging paradigm. Journal of Systems and Software, 103, 1–3. DOI: https://doi.org/10.1007/978-3-319-11283-1_1
[30] Zheng, Z., Chen, Y., & Li, X. (2018). Towards automated anomaly identification in large-scale systems. IEEE Transactions on Dependable and Secure Computing, 15(6), 1024–1037. DOI: https://doi.org/10.1109/TDSC.2017.2762673
[31] Zaharia, M., Chowdhury, M., Franklin, M. J., Shenker, S., & Stoica, I. (2010). Spark: Cluster computing with working sets. Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing, 10–10.
[32] He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284. DOI: https://doi.org/10.1109/TKDE.2008.239
[33] Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3(1), Article 9. DOI: https://doi.org/10.1186/s40537-016-0043-6
[34] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?” Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. DOI: https://doi.org/10.1145/2939672.2939778
[35] Davenport, T. H., & Harris, J. G. (2017). Competing on analytics: The new science of winning (Updated ed.). Harvard Business Review Press.
[36] Ghotra, B., McIntosh, S., & Hassan, A. E. (2015). Revisiting the impact of classification techniques on the performance of defect prediction models. Proceedings of the 37th International Conference on Software Engineering, 789–800. DOI: https://doi.org/10.1109/ICSE.2015.91
[37] Lane, N. D., & Georgiev, P. (2015). Can deep learning revolutionize mobile sensing? Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, 117–122. DOI: https://doi.org/10.1145/2699343.2699349
[38] Fu, Q., Zhu, J., Hu, W., Lou, J.-G., Ding, R., Lin, Q., Zhang, D., & Xie, T. (2014). Where do developers log? An empirical study on logging practices in industry. Proceedings of the 36th International Conference on Software Engineering, 24–33. DOI: https://doi.org/10.1145/2591062.2591175
[39] Bobbio, A., Portinale, L., Minichino, M., & Ciancamerla, E. (2001). Improving the analysis of dependable systems by mapping fault trees into Bayesian networks. Reliability Engineering & System Safety, 71(3), 249–260. DOI: https://doi.org/10.1016/S0951-8320(00)00077-6
[40] Zaharia, M., Xin, R. S., Wendell, P., Das, T., Armbrust, M., Dave, A., Meng, X., Rosen, J., Venkataraman, S., Franklin, M. J., et al. (2016). Apache Spark: A unified engine for big data processing. Communications of the ACM, 59(11), 56–65. DOI: https://doi.org/10.1145/2934664
[41] Satyanarayanan, M. (2017). The emergence of edge computing. Computer, 50(1), 30–39. DOI: https://doi.org/10.1109/MC.2017.9
[42] Xu, Y., Chen, N., Fernandez, A., Sinno, O., & Bhasin, S. (2015). From infrastructure to culture: A/B testing challenges in large scale social networks. Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2227–2236. DOI: https://doi.org/10.1145/2783258.2788602
[43] Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019). Software engineering for machine learning: A case study. Proceedings of the 41st International Conference on Software Engineering, 291–300. DOI: https://doi.org/10.1109/ICSE-SEIP.2019.00042
[44] Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. Proceedings of the 34th International Conference on Machine Learning, 1126–1135.
[45] McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & y Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 1273–1282.
[46] Pearl, J., Glymour, M., & Jewell, N. P. (2016). Causal inference in statistics: A primer. Wiley.
[47] Han, S., Mao, H., & Dally, W. J. (2016). Deep compression: Compressing deep neural networks with pruning, trained quantization and Huffman coding. International Conference on Learning Representations.
[48] Tao, F., Zhang, H., Liu, A., & Nee, A. Y. C. (2019). Digital twin in industry: State-of-the-art. IEEE Transactions on Industrial Informatics, 15(4), 2405–2415. DOI: https://doi.org/10.1109/TII.2018.2873186
[49] Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160. DOI: https://doi.org/10.1109/ACCESS.2018.2870052
[50] Kitchenham, B., & Brereton, O. P. (2013). A systematic review of systematic review process research in software engineering. Information and Software Technology, 55(12), 2049–2075. DOI: https://doi.org/10.1016/j.infsof.2013.07.010
[51] Biggio, B., & Roli, F. (2018). Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recognition, 84, 317–331. DOI: https://doi.org/10.1016/j.patcog.2018.07.023
[52] Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014). Power to the people: The role of humans in interactive machine learning. AI Magazine, 35(4), 105–120. DOI: https://doi.org/10.1609/aimag.v35i4.2513
[53] Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016). The ethics of algorithms: Mapping the debate. Big Data & Society, 3(2), 1–21. DOI: https://doi.org/10.1177/2053951716679679
[54] Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1), 107–113. DOI: https://doi.org/10.1145/1327452.1327492
[55] Murphy, K. P. (2012). Machine learning: A probabilistic perspective. MIT Press.
[56] Bishop, C. M. (2006). Pattern recognition and machine learning. Springer.
[57] Laprie, J.-C. (1992). Dependability: Basic concepts and terminology. Springer. DOI: https://doi.org/10.1007/978-3-7091-9170-5
[58] Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.-F., & Dennison, D. (2015). Hidden technical debt in machine learning systems. Advances in Neural Information Processing Systems, 28, 2503–2511.
[59] Gamma, E., Helm, R., Johnson, R., & Vlissides, J. (1994). Design patterns: Elements of reusable object-oriented software. Addison-Wesley.
[60] Leveson, N. (2011). Engineering a safer world: Systems thinking applied to safety. MIT Press. DOI: https://doi.org/10.7551/mitpress/8179.001.0001
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 International Journal of Computational and Experimental Science and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
