Abstract
Machine rule induction was examined on a difficult categorization problem by applying a Holland-style classifier system to a complex letter recognition task. A set of 20,000 unique letter images was generated by randomly distorting pixel images of the 26 uppercase letters from 20 different commercial fonts. The parent fonts represented a full range of character types including script, italic, serif, and Gothic. The features of each of the 20,000 characters were summarized in terms of 16 primitive numerical attributes. Our research focused on machine induction techniques for generating IF-THEN classifiers in which the IF part was a list of values for each of the 16 attributes and the THEN part was the correct category, i.e., one of the 26 letters of the alphabet. We examined the effects of different procedures for encoding attributes, deriving new rules, and apportioning credit among the rules. Binary and Gray-code attribute encodings that required exact matches for rule activation were compared with integer representations that employed fuzzy matching for rule activation. Random and genetic methods for rule creation were compared with instance-based generalization. The strength/specificity method for credit apportionment was compared with a procedure we call “accuracy/utility”.
Article PDF
Similar content being viewed by others
References
AckleyD.H., HintonG.E., & SejnowskiT.J. (1985). A learning algorithm for Boltzmann machines.Cognitive Science,9, 147–169.
Anderson, J.A. (1983). Cognitive and psychological computation with neural models.IEEE Transactions on Systems, Man, and Cybernetics, SMC-13, 799–815.
BookerL.B. (1988). Classifier systems that learn internal world models.Machine Learning,3, 161–192.
CaruanaR.A., & SchafferD. (1988). Representation and hidden bias: gray vs. binary coding for genetic algorithms.Proceedings of the Fifth International Conference on Machine Learning (pp. 153–161). Ann Arbor, MI: Morgan Kaufmann Publishers.
CharnessN. (1981). Aging and skilled problem-solving.Journal of Experimental Psychology: General,110, 21–38.
ChaseW.G., & SimonH.A. (1973). Perception in chess.Cognitive Psychology,4, 55–81.
DavisL., & YoungD.K. (1988). Classifier systems with Hamming weights.Proceedings of the Fifth International Conference on Machine Learning (pp. 162–173). Ann Arbor, MI: Morgan Kaufmann Publishers.
deGrootA.D. (1965).Thought and choice in chess. The Hague: Mouton, 2nd edition, 1978.
GoldbergD.E. (1989).Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley Publishing.
HollandJ.H. (1975).Adaptation in natural and artificial systems. Ann Arbor, MI: University of Michigan Press.
HollandJ.H. (1980). Adaptive algorithms for discovering and using general patterns in growing knowledge bases.International Journal of Policy Analysis and Information Systems,4, 217–240.
HollandJ.H. (1986). Escaping brittleness: The possibilities of general purpose machine learning algorithms applied to parallel rule-based systems. In R.S.Michalski, J.G.Carbonell, & T.M.Mitchell (Eds.),Machine learning: An artificial intelligence approach (Vol. II). San Mateo, CA: Morgan Kaufmann Publishers.
HollandJ.H., HolyoakK.J., NisbettR.E., & ThagardP.R. (1986).Induction: Processes of inference, learning, and discovery. Cambridge, MA: The MIT Press.
HuntE.B., MarinJ., & StoneP.J. (1966).Experiments in induction. New York: Academic Press.
QuinlanJ.R. (1979). Discovering rules by induction from large collections of examples. In D.Michie (Ed.),Expert systems in the micro electronic age. Edinburgh: Edinburgh University Press.
QuinlanJ.R. (1986). Induction of decision trees.Machine Learning,1, 81–106.
RobertsonJ.R. (1988). Population size in classifier systems.Proceedings of the Fifth International Conference on Machine Learning (pp. 142–152). Ann Arbor, MI: Morgan Kaufmann Publishers.
RumelhartD.E., & ZipserD. (1985). Feature discovery by competitive learning.Cognitive Science,9, 75–112.
StanfillC., & WaltzD. (1986). Toward memory-based reasoning.Communications of the ACM,29, 1213–1228.
SuttonR. (1988). Learning to predict by the method of temporal differences.Machine Learning,3, 9–44.
WilsonS.W. (1985). Knowledge growth in an artificial animal.Proceedings of an International Conference on Genetic Algorithms and Their Applications (pp. 16–23). Pittsburgh, PA: Lawrence Erlbaum Associates.
WilsonS.W. (1987). Classifier systems and the animat problem.Machine Learning,2, 199–228.
WilsonS.W. (1988). Bid competition and specificity reconsidered.Complex Systems,2, 705–723.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Frey, P.W., Slate, D.J. Letter recognition using Holland-style adaptive classifiers. Mach Learn 6, 161–182 (1991). https://doi.org/10.1007/BF00114162
Issue date:
DOI: https://doi.org/10.1007/BF00114162
