Hostname: page-component-5db58dd55d-htx7c Total loading time: 0 Render date: 2026-05-25T16:03:58.434Z Has data issue: false hasContentIssue false
  • English
  • Français

A New Solution to the Puzzle of Simplicity

Published online by Cambridge University Press:  01 January 2022

Kevin T. Kelly
Get access Rights & Permissions [Opens in a new window]

Abstract

Explaining the connection, if any, between simplicity and truth is among the deepest problems facing the philosophy of science, statistics, and machine learning. Say that an efficient truth finding method minimizes worst case costs en route to converging to the true answer to a theory choice problem. Let the costs considered include the number of times a false answer is selected, the number of times opinion is reversed, and the times at which the reversals occur. It is demonstrated that (1) always choosing the simplest theory compatible with experience, and (2) hanging onto it while it remains simplest, is both necessary and sufficient for efficiency.

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Aho, A., Hopcroft, J., and Ullman, J. (1974), The Design and Analysis of Computer Algorithms. New York: Addison-Wesley.Google Scholar
Akaike, H. (1973), “Information Theory and an Extension of the Maximum Likelihood Principle”, in Petrov, B. N. and Csaki, F. (eds.), The Second International Symposium on Information Theory. Budapest: Akadémiai Kiadó, 267–281.Google Scholar
Bonjour, L. (1985), The Structure of Empirical Knowledge. Cambridge, MA: Harvard University Press.Google Scholar
Carnap, R. (1950), Logical Foundations of Probability. Chicago: University of Chicago Press.Google Scholar
Daley, R., and Smith, C., (1986), “On the Complexity of Inductive Inference”, On the Complexity of Inductive Inference 69:12–40.Google Scholar
Duda, R., Stork, D., and Hart, P. (2000), Pattern Classification, Vol. 1. New York: Wiley.Google Scholar
Forster, M., and Sober, E. (1994), “How to Tell When Simpler, More Unified, or Less Ad Hoc Theories Will Provide More Accurate Predictions”, How to Tell When Simpler, More Unified, or Less Ad Hoc Theories Will Provide More Accurate Predictions 45:1–35.Google Scholar
Friedman, M. (1983), Foundations of Space-Time Theories: Relativistic Physics and Philosophy of Science. Princeton, NJ: Princeton University Press.Google Scholar
Gärdenfors, P. (1988), Knowledge in Flux. Cambridge, MA: MIT Press.Google Scholar
Giere, R. (1985), “Philosophy of Science Naturalized,” Philosophy of Science 52:331–356.CrossRefGoogle Scholar
Glymour, C. (1980), Theory and Evidence. Princeton, NJ: Princeton University Press.Google Scholar
Gold, E. (1978), “Language Identification in the Limit”, Language Identification in the Limit 10:447–474.Google Scholar
Goodman, N. (1983), Fact, Fiction, and Forecast. Cambridge, MA: Harvard University Press.Google Scholar
Harman, G. (1965), “The Inference to the Best Explanation”, The Inference to the Best Explanation 74:88–95.Google Scholar
Jain, S., Osherson, D., Royer, J., and Sharma, A. (1999), Systems That Learn. Cambridge, MA: MIT Press.CrossRefGoogle Scholar
Kelly, K. (2002), “Efficient Convergence Implies Ockham’s Razor”, in Delrieux, Claudio (ed.), Proceedings of the 2002 International Workshop on Computational Models of Scientific Reasoning and Applications. Bogart, GA: CSREA.Google Scholar
Kelly, K. (2004), “Justification as Truth-Finding Efficiency: How Ockham’s Razor Works”, Justification as Truth-Finding Efficiency: How Ockham’s Razor Works 14:485–505.Google Scholar
Kelly, K. (2007), “Ockham’s Razor, Empirical Complexity, and Truth-Finding Efficiency”, Theoretical Computer Science, 270–289.CrossRefGoogle Scholar
Kelly, K. (2008), “Ockham’s Razor, Truth, and Information,” in Benthem, J. Van and Adriaans, P. (eds.), Philosophy of Information. Amsterdam: Elsevier, forthcoming.Google Scholar
Kelly, K., and Glymour, C. (2004), “Why Probability Does Not Capture the Logic of Scientific Justification”, in Hitchcock, C. (ed.), Contemporary Debates in the Philosophy of Science. Oxford: Blackwell, 94–114.Google Scholar
Leibniz, G. W. ([1714] 1875), Monadologie, in Loemker, L. E. (ed.), Die Philosophischen Schriften von G. W. Leibniz, Vol. 4. Berlin: Gerhardt, 607–623.Google Scholar
Popper, K. (1968), The Logic of Scientific Discovery. New York: Harper.Google Scholar
Putnam, H. (1965) “Trial and Error Predicates and a Solution to a Problem of Mostowski”, Trial and Error Predicates and a Solution to a Problem of Mostowski 30:49–57.Google Scholar
Rissanen, J. (1983), “A Universal Prior for Integers and Estimation by Minimum Description Length”, A Universal Prior for Integers and Estimation by Minimum Description Length 11:416–431.Google Scholar
Rosenkrantz, R. (1983), “Why Glymour Is a Bayesian”, in Earman, J. (ed.), Testing Scientific Theories. Minneapolis: University of Minnesota Press, 69–98.Google Scholar
Salmon, W. (1967), The Logic of Scientific Inference. Pittsburgh: University of Pittsburgh Press.CrossRefGoogle Scholar
Schulte, O. (1999), “Means-Ends Epistemology”, Means-Ends Epistemology 50:1–31.Google Scholar
Schulte, O. (2000), “Inferring Conservation Principles in Particle Physics: A Case Study in the Problem of Induction”, Inferring Conservation Principles in Particle Physics: A Case Study in the Problem of Induction 51:771–806.Google Scholar
Spirtes, P., Glymour, C., and Scheines, R. (2000), Causation, Prediction, and Search. Cambridge, MA: MIT Press.Google Scholar
Spirtes, P., and Zhang, J. (2003), “Strong Faithfulness and Uniform Consistency in Causal Inference”, in Meek, Christopher and Kjærulff, Uffe (eds.), Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence. San Mateo, CA: Kaufmann, 632–639.Google Scholar
van Fraassen, B. (1981), The Scientific Image. Oxford: Clarendon.Google Scholar
Wasserman, L. (2004), All of Statistics: A Concise Course in Statistical Inference. New York: Springer.CrossRefGoogle Scholar