An Empirical Study of Qualities of Association Rules from a Statistical View Point
2
조회
0
다운로드
0
인용
0
참고문헌
키워드
초록
Minimum support and confidence have been used as criteria for generating association rules in all association rule mining algorithms. These criteria have their natural appeals, such as simplicity; few researchers have suspected the quality of generated rules. In this paper, we examine the rules from a more rigorous point of view by conducting statistical tests. Specifically, we use contingency tables and chi-square test to analyze the data. Experimental results show that one third of the association rules derived based on the support and confidence criteria are not significant, that is, the antecedent and consequent of the rules are not correlated. It indicates that minimum support and minimum confidence do not provide adequate discovery of meaningful associations. The chi-square test can be considered as an enhancement or an alternative solution.
Abstract
Minimum support and confidence have been used as criteria for generating association rules in all association rule mining algorithms. These criteria have their natural appeals, such as simplicity; few researchers have suspected the quality of generated rules. In this paper, we examine the rules from a more rigorous point of view by conducting statistical tests. Specifically, we use contingency tables and chi-square test to analyze the data. Experimental results show that one third of the association rules derived based on the support and confidence criteria are not significant, that is, the antecedent and consequent of the rules are not correlated. It indicates that minimum support and minimum confidence do not provide adequate discovery of meaningful associations. The chi-square test can be considered as an enhancement or an alternative solution.
참고문헌 (21)
- Agrawal, R, "Mining Association Rules Between Sets of Items in Large Databases", Proc. of the ACM-SIGMOD Conf. on Management of Data, pp.207~216 (1993)
- Agrawal R, "Fast algorithms for Mining Association Rules", Proc. of the 20th VLDB Conference, pp.487~499 (1994)
- Fast Algorithms for Mining Association Rules, "Agrawal, R", ~, pp.IBM Research Report RJ9839 (1994)
- Bayardo, R. J., "Mining the Most Interesting Rules", Proc. of the Fifth ACM SIGKDD Conf. on Knowledge Discovery and Data Mining, pp.145~154 (1999)
- Bayardo, R., "Constraint-Based Rule Mining in Large, Dense Databases", Proc. of the 15th Int'l Conf. on Data Engineering, pp.188~197 (1999)
- Brin, S, "Beyond Market Basket: Generalizing Association Rules to Correlations", SIGMOD-97, pp.265~276 (1997)
- Brin, S, "Dynamic Itemset Counting and Implication Rules for Market Basket Data", Proc. of the 1997 ACM-SIGMOD Int’l Conf. on the Management of Data, pp.255~267 (1997)
- Ganti, V., Gebrke, "Mining Very Large Databases", Computer, 32(8), pp.38~45 (1999)
- Glass, G. V., "Statistical Methods in Education and Psychology" (1984), Prentice Hall
- Gokhale, D. V., "The Information in Contingency Tables" (1978), Marcel Dekker Inc.
- Han, J, "Discovery of multiple-level association rules from large databases", VLDB-95
- Liu B, "Pruning and Summarizing the Discovered Associations", Proc. of the ACM SIGKDD Int’l Conference on Knowledge Discovery & Data Mining (1999)
- Liu B., "Mining Association Rules with Multiple Minimum Supports", Proc. of the ACM SIGKDD Int’l Conference on Knowledge Discovery & Data Mining (1999)
- Liu B, "Mining Interesting Knowledge Using DM-II", Proc. of the ACM SIGKDD Int’l Conference on Knowledge Discovery & Data Mining (1999)
- Mason, R. D., "STATISTICS: An Introduction" (1998), Duxbury Press
- Park, J. S., "An Effective Hash Based Algorithm for Mining Association Rules", Proc. of SIGMOD Conf. on the Management of Data, pp.175~186 (1995)
- Srikant, R., "Mining Generalized Association Rules", Proc. of the 21st Int'l Conf. on VLDB, pp.407~419 (1995)
- Mining Generalized Association Rules, "Srikant, R", ~, pp.IBM Research Report RJ9963 (1995)
- Srikant, R., "Mining Association Rules with Item Constraints", Proc. of the Third Int'l Conf. on Knowledge Discovery in Databases and Data Mining, pp.67~73 (1997)
- Toivonen H, "Sampling Large Databases for Association Rules", Proc. of the 22th VLDB Conference, pp.134~144 (1996)
- Zaki, M J, "New Algorithms for Fast Discovery of Association Rules", Proc. of the Third Int'l Conf. on Knowledge Discovery in Databases and Data Mining, pp.283~286 (1997)