Vol. 1 No. 4 (2026)

Articles

Metamorphic Testing of Watermark and Window Semantics in Stream-Query Assistants

Published 2026-08-29

  • Paula Reyes

Paula Reyes
Electronics and Telecommunications Department, Faculty of Engineering, Universidad de Antioquia, Medellín, Colombia

Language, Knowledge and Intelligent Systems, Volume 1, Issue 4, 2026 cover

Abstract

The proliferation of real-time data analytics has positioned stream-processing systems and their associated query assistants as critical components of modern computational infrastructures. These systems process continuous data flows using complex temporal constructs, most notably window semantics for data aggregation and watermark semantics for handling out-of-order events. Testing the correctness of these semantics presents a significant challenge due to the lack of a test oracle, as the expected output for arbitrary continuous streams is often non-deterministic or practically impossible to determine beforehand. To address this oracle problem, this paper proposes a comprehensive metamorphic testing framework tailored specifically for watermark and window semantics in stream-query assistants. By defining a robust set of metamorphic relations that capture the essential properties of stream transformations, we enable automated test case generation and output verification without requiring predefined expected results. The framework systematically perturbs input streams by altering event times, arrival orders, and watermark thresholds to evaluate the consistency of the underlying query processors. Extensive experiments on representative stream datasets demonstrate the efficacy of the proposed approach in revealing subtle semantic violations and implementation anomalies in popular stream processing engines. The results indicate that metamorphic testing is highly effective in ensuring the reliability and robustness of complex temporal data processing paradigms.

Keywords

Metamorphic Testing, Stream Processing, Watermark Semantics, Window Functions

PDF

References

  1. 1. Zhao, R., Tang, J., Zeng, W., Chen, Z., & Zhao, X. (2024, October). Zero-shot knowledge graph question generation via multi-agent llms and small models synthesis. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (pp. 3341-3351).
  2. 2. Huang, Y., Thede, L., Mancini, M., Xu, W., & Akata, Z. (2025, September). Investigating structural pruning and recovery techniques for compressing multimodal large language models: An empirical study. In DAGM german conference on pattern recognition (pp. 320-336). Cham: Springer Nature Switzerland.
  3. 3. Zhang, W., & Pei, S. (2026). Predictive prefetching for retrieval-augmented generation. Proceedings of Machine Learning Research. The Forty-third International Conference on Machine Learning, ICML 2026. https://par.nsf.gov/biblio/10686293
  4. 4. Sang, Y. (2025, July). Towards explainable rag: Interpreting the influence of retrieved passages on generation. In 2025 4th International Conference on Robotics, Artificial Intelligence and Intelligent Control (RAIIC) (pp. 397-400). IEEE.
  5. 5. Sang, Y. (2025, October). AutoCrit: A Meta-Reasoning Framework for Self-Critique and Iterative Error Correction in LLM Chains-of-Thought. In 2025 6th International Conference on Machine Learning and Computer Application (ICMLCA) (pp. 1177-1180). IEEE.
  6. 6. Tang, J., Wang, Z., Gong, Z., Yu, J., Zhu, X., & Yin, J. (2025, April). Multi-grained query-guided set prediction network for grounded multimodal named entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 39, No. 24, pp. 25246-25254).
  7. 7. Tang, J., Yang, Y., Yu, J., Wang, Z. X., Liang, H., Yao, L., & Yin, J. (2025, November). Unco: Uncertainty-driven collaborative framework of large and small models for grounded multimodal ner. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (pp. 7644-7662).
  8. 8. Zhang, Y., Zhao, M., Zhang, Y., & Cheung, Y. M. (2025). Trending applications of large language models: A user perspective survey. IEEE Transactions on Artificial Intelligence.
  9. 9. Jiang, J., Yang, P., Zhang, R., & Liu, F. (2026, July). Towards efficient large language model serving: A survey on system-aware kv cache optimization. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 38450-38476).
  10. 10. Zhu, D., Xie, C., Zhang, H., Wei, Z., Wang, Z., Shi, J., ... & Xie, Q. (2026). Standalone LLM and a Pre-specified Agentic Pipeline for Explaining ICU Mortality Predictions: a Feasibility Study on the eICU Demo Dataset. arXiv preprint arXiv:2608.26109.
  11. 11. Qiu, Z., Qiu, K., Lyu, H., Xiong, W., & Luo, J. (2024, December). Semantics preserving emoji recommendation with large language models. In 2024 IEEE International Conference on Big Data (BigData) (pp. 7131-7140). IEEE.
  12. 12. Wang, H., Xu, Q., Liu, C., Wu, J., Lin, F., & Chen, W. (2026, April). Emergent hierarchical reasoning in llms through reinforcement learning. In International Conference on Learning Representations (Vol. 2026, pp. 74519-74543).
  13. 13. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems.
  14. 14. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018). Deep contextualized word representations. In Proceedings of NAACL-HLT.
  15. 15. Cho, K., van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of EMNLP.
  16. 16. Zhao, R., Xu, D., Jian, S., Tan, T., Sun, X., & Zhang, W. (2023, July). Quadratic Exponential Decrease Roll-Back: An Efficient Gradient Update Mechanism in Proximal Policy Optimization. In 2023 2nd International Conference on Machine Learning, Cloud Computing and Intelligent Mining (MLCCIM) (pp. 65-70). IEEE.
  17. 17. Li, Q., Ye, Q., Zhang, N., Zhang, W., & Hu, F. (2025). Digital-twin-enabled industrial IoT: Vision, framework, and future directions. IEEE Wireless Communications, 32(6), 173-181.
  18. 18. Li, S. (2025). Momentum, volume and investor sentiment study for us technology sector stocks—A hidden markov model based principal component analysis. PloS one, 20(9), e0331658.
  19. 19. Li, X., Wang, P., Li, G., Ni, L., & Zhang, Y. (2023). Design of interface circuits and lightweight PUF for TMR sensors. IEEE Sensors Journal, 23(11), 11754-11761.
  20. 20. Li, X., Yang, F., Chen, L., & Cai, H. (2016, July). Saliency transfer: An example-based method for salient object detection. In IJCAI (pp. 3411-3417).
  21. 21. Zhao, J., Zhang, C., Qin, M., & Yang, P. (2025). QuantFactor REINFORCE: mining steady formulaic alpha factors with variance-bounded REINFORCE. IEEE Transactions on Signal Processing, 73, 2448-2463.
  22. 22. Yang, Z., Hu, D., Guo, Q., Zuo, L., & Ji, W. (2023). Visual E 2 C: AI-driven visual end-edge-cloud architecture for 6G in low-carbon smart cities. IEEE Wireless Communications, 30(3), 204-210.
  23. 23. Lin, Y. (2024). Design of urban road fault detection system based on artificial neural network and deep learning. Frontiers in neuroscience, 18, 1369832.
  24. 24. Zhang, Y., Huang, Z., Zhao, M., Zhang, C., Lu, Y., Ji, Y., ... & Zeng, A. (2025). Learning unbiased cluster descriptors for interpretable imbalanced concept drift detection. IEEE Transactions on Emerging Topics in Computational Intelligence.
  25. 25. Li, T., Li, X., & Qu, Y. (2025). Autoformer-Based Sales and Inventory Forecasting for Cross-Border E-Commerce: A Time Series Deep Learning Approach.
  26. 26. Li, X., Lin, Y., He, W., Liu, R., Oliveira, A. L., Qian, T., ... & Hon, C. (2025). Enhancing the interpretation of spirometry: Joint utilization of n-order adaptive fourier decomposition and deep learning techniques. IEEE Transactions on Instrumentation and Measurement, 74, 1-14.