Information Theory

We information-theoretically reformulate two measures of capacity from statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. We show these capacity measures count the number of hypotheses about a dataset that a learning algorithm falsifies when it finds the classifier in its repertoire minimizing empirical risk. It then follows from that the future performance of predictors on unseen data is controlled in part by how many hypotheses the learner falsifies. As a corollary we show that empirical VC-entropy quantifies the message (...) length of the true hypothesis in the optimal code of a particular probability distribution, the so-called actual repertoire. (shrink)

The notion of a partition on a set is mathematically dual to the notion of a subset of a set, so there is a logic of partitions dual to Boole's logic of subsets (Boolean logic is usually mis-specified as "propositional" logic). The notion of an element of a subset has as its dual the notion of a distinction of a partition (a pair of elements in different blocks). Boole developed finite logical probability as the normalized counting measure on elements of (...) subsets so there is a dual concept of logical entropy which is the normalized counting measure on distinctions of partitions. Thus the logical notion of information is a measure of distinctions. Classical logical entropy naturally extends to the notion of quantum logical entropy which provides a more natural and informative alternative to the usual Von Neumann entropy in quantum information theory. The quantum logical entropy of a post-measurement density matrix has the simple interpretation as the probability that two independent measurements of the same state using the same observable will have different results. The main result of the paper is that the increase in quantum logical entropy due to a projective measurement of a pure state is the sum of the absolute squares of the off-diagonal entries ("coherences") of the pure state density matrix that are zeroed ("decohered") by the measurement, i.e., the measure of the distinctions ("decoherences") created by the measurement. (shrink)

Contemporary paradigms in artificial intelligence, cognitive science, and philosophy often equate intelligence with efficient compression-be it via statistical learning, formal abstraction, or narrative simplification. This paper challenges that assumption by identifying and formalizing a foundational paradox: the pursuit of maximal simplicity can annihilate meaning. We define this tension as the Non Grata Problem-the systemic exclusion of semantically rich but compression-resistant thoughts and tokens. Through a synthesis of information theory, predictive processing, game theory, and phenomenology, we introduce a new framework for (...) understanding cognition as a dynamic Nash equilibrium between multiple cognitive agents, each optimizing different compression strategies. We propose the Semantic Incompleteness Theorem, showing that no compressed formal system can fully capture the expressivity of natural language. Further, we demonstrate the Compression-Tail Paradox, revealing how aggressively compressed probabilistic models produce blind spots for high-impact outliers-so-called Black Swans. To integrate these insights, we construct the notion of a Transcendent Probability Space, a formal model for state-space overflow and the epistemology of surprise. By tracing these mathematical, philosophical, and aesthetic arguments through diverse domains-from haiku and hyperrealism to semantic dementia and LLMs-we argue for an ecology of understanding that honors the ineffable, irreducible, and richly entangled nature of human meaning. Compression, while essential, must be decentered from its absolutist role; the razor must not sever the roots of cognition. -/- . (shrink)

This paper explores the concept of information beyond its common role as descriptive data. It suggests that information operates as an intrinsic structure, influencing and defining systems across different dimensions and contexts, much like a universal “instruction set.” From societal bonds to natural phenomena, information is not passive but dynamic and often self-propagating, like a force. This document examines the function, autonomy, and density of information, considering it as fundamental to existence itself.

NOTE: This work is conceptual and exploratory. It outlines early motivations and heuristic ideas, and should not be taken as identical to the phenomenological models currently under development — including both the structural Δw approach and the scalar-field formulation used in technical perturbation and MCMC analyses. It is intended as background context rather than a formal physical model. -/- This paper introduces Relational Gravity, an extension to general relativity that incorporates structured information, modeled through quantum entanglement entropy, as a source (...) of spacetime curvature. We modify the stress–energy tensor to include an informational component: Tµν = T matter µν + T radiation µν + T info µν, where T info µν is derived from the variation ofentanglement entropy with respect to the metric. This framework reinterprets dark matteras localized informational density bound to galactic topology and dark energy as the expansive effect of growing cosmic entropy. It addresses the vacuum catastrophe by treating local (Higgs) and global (dark energy) vacua as scale-separated expressions of relational structure. Consciousness is considered as a high-density form of informational organization, potentially influencing quantum measurement, but the core emphasis is on physical testability. We derive the formalism, discuss implications for quantum mechanics and cosmology, and propose four sharpened quantitative predictions linked to ongoing experiments (e.g., JWST, EHT, quantum optics). This approach bridges general relativity, quantum information theory, and emergent gravity, offering a testable path toward unification. -/- Also archived on Zenodo for long term preservation (DOI: 10.5281/zenodo.17377368) -/- --- -/- Foundational Framework Statement Relational Gravity (Hummel 2025) introduces the informational-geometric framework in which spacetime curvature is proportional to coherent information density (???? = κ I₍coh₎). This publication constitutes the conceptual and ontological foundation of the Relational Gravity model. All subsequent derivations and observational tests extend this work. -/- Linked works (non-clickable addresses): Technical Addendum (v1.1): doi: 10.5281/zenodo.17435692 Short Paper: Relational Gravity: Coherence from Atoms to Cosmos. A Unified View of the Dark Sector doi: 10.5281/zenodo.17548415 -/- License: CC BY 4.0 Author ORCID: 0009-0005-6192-3372 -/- . (shrink)

This paper proposes a novel model for understanding consciousness by envisioning Brahman—the ultimate, universal consciousness in Vedanta—as an infinitely layered JSON (JavaScript Object Notation) structure. In this model, each aspect of reality, from individual thoughts to timelines and entire universes, is represented as a property within this universal JSON object. By integrating the Vedantic concepts of Atman (individual self) and Brahman (universal consciousness) with ideas from quantum mechanics and information theory, this model suggests that all possible experiences, thoughts, and realities (...) exist within Brahman, ready to be accessed by individual consciousness. This framework offers a fresh way to think about reality, creativity, and free will, proposing that every possible thought or action already exists as a “node” within Brahman’s JSON, and that we simply “select” which part to experience. (shrink)

The Phrenological Argument, also known as God's Parapraxis, is a new provocative philosophical argument that showcases God's Mental State of Inadequacy in his creation of the Human Mind. Lawsin* argues that if God has a perfect mind flawlessly capable of knowing things in advance, then he should have anticipated ahead of time the repercussions of creating a human mind that cannot fathom Truth and Reality. But since he created a mind incapable of knowing what is real and what is true, (...) then it shows God carries an erroneous lapse somewhere in his memory. This mental erracity offers good solid evidence that proves God is not after all all-knowing. -/- The basic form of the argument is: 1. God created the mind 2. The mind cannot detect truth and reality 3. therefore, God will never be known. -/- A modified version: 1. If God created the mind for men to know him 2. But god failed to foresee his parapraxis 3. Then, god is not all-knowing. -/- A follow-up variation: 1. If God is real 2. But the mind can't detect reality 3. Therefore, the mind can't detect god. -/- Furthermore, man created the idea of god. Like all ideas, the idea of god is also nothing but an assumption, a guesswork, a supposition. For example, Zero and One are abstract concepts. They don't exist in the natural world. They are not naturally inherent. They only exist in the mind by assumption. Mathematically, both words are called numerals by definition. When Zero is represented with the symbol 0 and One with 1, technically, the words become numbers by association. Numbers are the assumed physical representations of abstract numerals. By symbolic representation, both digits now exist outside the mind - the physical world - the world outside of ourselves - the inherent world that existed a long time independently before the mind. However, although 0 & 1 are created by definition, association, representation, and assumption, are these numbers now solid objects or physical materials? If I write 0 and 1 on a paper, are they materially or physically real now? Can we consider the written numbers as proof of their existence? Can the numbers be proven as solid evidence prescribed by the scientific method? How can we validate the paper's evidence to be true, false, valid, or real? Do they really exist or are the numbers still abstract, imaginary, or imagined? If they are still abstract, therefore all ideas man has thought of or conceptualized are all assumptions, guesswork, or nothing but merely suppositions (The Lawsin Conjecture/Codexation Dilemma). (shrink)

Semantic Information conveyed by daily language has been researched for many years; yet, we still need a practical formula to measure information of a simple sentence or prediction, such as “There will be heavy rain tomorrow”. For practical purpose, this paper introduces a new formula, Semantic Information Formula (SIF), which is based on L. A. Zadeh’s fuzzy set theory and P. Z. Wang’s random set falling shadow theory. It carries forward C. E. Shannon and K. Popper’s thought. The fuzzy set’s (...) probability defined by Zadeh is treated as the logical probability sought by Popper, and the membership grade is treated as the truth-value of a proposition and also as the posterior logical probability. The classical relative information formula (Information=log(Posterior probability / Prior probability) is revised into SIF by replacing the posterior probability with the membership grade and the prior probability with the fuzzy set’s probability. The SIF can be explained as “Information=Testing severity – Relative square deviation” and hence can be used as Popper's information criterion to test scientific theories or propositions. The information measure defined by the SIF also means the spared codeword length as the classical information measure. This paper introduces the set-Bayes’ formula which establishes the relationship between statistical probability and logical probability, derives Fuzzy Information Criterion (FIC) for the optimization of semantic channel, and discusses applications of SIF and FIC in areas such as linguistic communication, prediction, estimation, test, GPS, translation, and fuzzy reasoning. Particularly, through a detailed example of reasoning, it is proved that we can improve semantic channel with proper fuzziness to increase average semantic information to reach its upper limit: Shannon mutual information. (shrink)

!Author’s Note (Update - June 18, 2025) -/- Subsequent to the completion of this paper, a widely discussed study by Shojaee et al. at Apple (The Illusion of Thinking, 2025) presented empirical evidence that several advanced reasoning models (e.g., Claude 3.7 Sonnet, DeepSeek-R1) exhibit a collapse in reasoning effort and accuracy as task complexity increases — despite sufficient inference budget. Notably, the observations they report closely mirror the predictions derived in Chapters 6 and 7 of this paper, particularly regarding entropy (...) divergence in heavy-tailed decision spaces and the increased instability introduced by deeper model architectures. While developed independently and prior to these results, the framework presented here may be seen as a theoretical explanation for the empirical phenomena documented in their study. -/- Abstract -/- This paper presents a second, independent proof that Artificial General Intelligence (AGI) is mathematically impossible. -/- While Part 1 established the Infinite Choice Barrier (ICB) through computability theory, this paper presents an information-theoretic proof that general algorithmic reasoning faces a fundamental structural limit — the Infinite Choice Barrier (ICB). In decision spaces governed by heavy-tailed distributions with tail exponent \alpha \leq 1, entropy does not converge, and additional information increases uncertainty rather than reducing it. We define this phenomenon as Information Opens, Entropy Rises (IOpenER) and show that such spaces are undecidable by algorithmic systems, regardless of compute scale or architectural depth. We also introduce the Layer Hypothesis, which predicts that deeper models produce more semantic divergence under structural instability. Taken together, these results offer a formal limit on the reasoning capacity of current AI systems. Notably, key predictions derived here have since been empirically confirmed in Shojaee et al. (Apple, 2025), which documents collapse behaviors in frontier reasoning models consistent with this framework. (Note: Sections 2.-4. are identical to those in the first paper "AGI is Impossible - Here is The Proof") -/- . (shrink)

This paper is a mathematical consolidation of my papers on the impossibility of AGI. It presents a triangulated impossibility proof for Artificial General Intelligence (AGI), defined as an algorithmic system capable of solving a sufficiently broad class of human problems at human-level competence. Three formally independent but converging mathemati- cal arguments-based in computability theory, information theory, and algorithmic complexity- jointly demonstrate that AGI, as such, cannot exist. -/- This boundary, named the Infinite Choice Barrier (ICB), is neither technical nor architectural. (...) It is structural by its nature. It marks a categorical limitation on algorithmic cognition. -/- Philosophical groundwork and contextual development of the formal proof are explored in: Part I: AGI is Impossible - Here is The Proof Part II: AGI is Mathematically Impossible 2: When Entropy Returns (Both papers available on Philpapers) A third part is still to be finalized and will be uploaded, the formal part is already in this paper. (shrink)

This paper offers a first formal and philosophical refoundation of a new critique of Artificial General Intelligence (AGI), introducing the Infinite-Choice Barrier—a structural impossibility theorem demonstrating that no algorithmic system can achieve ε-optimal performance across irreducibly infinite decision spaces, nor autonomously generate novel semantic frames. Drawing on Gödel’s incompleteness theorem, Turing’s Halting Problem, and Kantian epistemology, the argument identifies the epistemic boundary of algorithmic cognition. -/- Three corollaries ground the framework: • Semantic Closure — systems are confined to their native (...) representational vocabulary. • No Frame Innovation — genuine conceptual novelty is non-computable from within. • Statistical Breakdown in Fat-Tailed Domains — inference fails under radical uncertainty. -/- These are not empirical ceilings but formal constraints that reveal where machine reasoning must collapse—not due to training limits, but from foundational undecidability. The paper concludes with a taxonomy of irreducible human decision types and a philosophical reflection on the boundary between human and artificial cognition. (shrink)

An attempt to vindicate naive set theory by postulating a universal set V which is describable in two distinct description languages: predicative and extensional. The extensional description of a set consists of describing all its elements whereas its predicative description consists of describing what sets it is an element of. -/- Extensionally described V has an uncapturable description length, akin to its cardinality. But predicatively described, in virtue of being the set that is not contained in any set whatsoever, V (...) has a minimal description length, a counterbalance to its cardinality. -/- Descriptive efficiency can genuinely be attributed to V (achieved in a third description language) only if it contains predicatively indiscernible but extensionally discernible sets. (shrink)

The Universal Variational Paradigm (UVP) defines reality as a self-consistent variational process of distinctions. All physical laws, stable structures, and temporal evolution arise from a single informational functional — the Fisher information measure. From three irreducible axioms — difference, stationarity, and non-closure — UVP derives quantum mechanics, general relativity, and the arrow of time as expressions of the same universal law. The theory unifies physics, biology, and consciousness under one variational geometry where being is the stable evolution of difference.

There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it [1]. The second, Shannon information, takes events as belonging to ensembles and quantifies the information resulting from observing the given event in terms of the number of alternate events that have been ruled out [2]. The third, statistical learning (...) theory, has introduced measures of capacity that control (in part) the expected risk of classifiers [3]. These capacities quantify the expectations regarding future data that learning algorithms embed into classifiers. Solomonoff and Hutter have applied algorithmic information to prove remarkable results on universal induction. Shannon information provides the mathematical foundation for communication and coding theory. However, both approaches have shortcomings. Algorithmic information is not computable, severely limiting its practical usefulness. Shannon information refers to ensembles rather than actual events: it makes no sense to compute the Shannon information of a single string – or rather, there are many answers to this question depending on how a related ensemble is constructed. Although there are asymptotic results linking algorithmic and Shannon information, it is unsatisfying that there is such a large gap – a difference in kind – between the two measures. This note describes a new method of quantifying information, effective information, that links algorithmic information to Shannon information, and also links both to capacities arising in statistical learning theory [4, 5]. After introducing the measure, we show that it provides a non-universal analog of Kolmogorov complexity. We then apply it to derive basic capacities in statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. A nice byproduct of our approach is an interpretation of the explanatory power of a learning algorithm in terms of the number of hypotheses it falsifies [6], counted in two different ways for the two capacities. We also discuss how effective information relates to information gain, Shannon and mutual information. (shrink)

given at the 2007 Formal Epistemology Workshop at Carnegie Mellon June 2nd. Good compression must track higher vs lower probability of inputs, and this is one way to approach how simplicity tracks truth.

The Kolmogorov-Sinai entropy is a fairly exotic mathematical concept which has recently aroused some interest on the philosophers’ part. The most salient trait of this concept is its working as a junction between such diverse ambits as statistical mechanics, information theory and algorithm theory. In this paper I argue that, in order to understand this very special feature of the Kolmogorov-Sinai entropy, is essential to reconstruct its genealogy. Somewhat surprisingly, this story takes us as far back as the beginning of (...) celestial mechanics and through some of the most exciting developments of mathematical physics of the 19th century. (shrink)

The present paper looks at one of the most thorough articles on the intelligence of GPT, research conducted by engineers at Microsoft. Although there is a great deal of value in their work, I will argue that, for familiar philosophical reasons, their methodology, ‘Black-box Interpretability’ is wrongheaded. But there is a better way. There is an exciting and emerging discipline of ‘Inner Interpretability’ (also sometimes called ‘White-box Interpretability’) that aims to uncover the internal activations and weights of models in order (...) to understand what they represent and the algorithms they implement. In my view, a crucial mistake in Black-box Interpretability is the failure to appreciate that how processes are carried out matters when it comes to intelligence and understanding. I can’t pretend to have a full story that provides both necessary and sufficient conditions for being intelligent, but I do think that Inner Interpretability dovetails nicely with plausible philosophical views of what intelligence requires. So the conclusion is modest, but the important point in my view is seeing how to get the research on the right track. Towards the end of the paper, I will show how some of the philosophical concepts can be used to further refine how Inner Interpretability is approached, so the paper helps draw out a profitable, future two-way exchange between Philosophers and Computer Scientists. (shrink)

Scholarship on the politics of new media widely assumes that communication functions as a sufficient conceptual paradigm for critically assessing new media politics. This article argues that communication-centric analyses fail to engage the politics of information itself, limiting information only to its consequences for communication, and neglecting information as it reaches into our selves, lives, and actions beyond the confines of communication. Furthering recent new media historiography on the “information theory” of Shannon and Wiener, the article reveals both the primacy (...) of communication in midcentury information theory, and also a striking resonance between these postwar communication theories and Habermas’s more recent communicative theory of democracy. To achieve a critical perspective beyond communication, the article proposes a media genealogy of the politics of subjects as a methodology for developing an analysis of how information formats us as subjects of data. (shrink)

In spite of being ubiquitous in life sciences, the concept of information is harshly criticized. Uses of the concept other than those derived from Shannon's theory are denounced as pernicious metaphors. We perform a computational experiment to explore whether Shannon's information is adequate to describe the uses of said concept in commonplace scientific practice. Our results show that semantic sequences do not have unique complexity values different from the value of meaningless sequences. This result suggests that quantitative theoretical frameworks do (...) not account fully for the complex phenomenon that the term “information” refers to. We propose a restructuring of the concept into two related, but independent notions, and conclude that a complete theory of biological information must account completely not only for both notions, but also for the relationship between them. (shrink)

Information Theory of Abiogenesis: Life's origin is a mystery. Science may never note for sure how life originated. However, engineering MAY create life. Then one MAY deduce information theory(more specifically creating a phenotype, then a corresponding replicator) is at least how certain life was created.

Review by: Jan Reimann The Bulletin of Symbolic Logic, Volume 19, Issue 3, Page 397-399, September 2013.

The emergence of recursive coherence as a formal system-level invariant parallels and is catalyzed by the proliferation of generative AI, particularly large language models (LLMs) such as GPT-3 and GPT-4. These models, scaled to billions of interactions, generated recursive tension: human users constructed layered prompt chains, real-time feedback loops, and tool-augmented cognitive architectures that exposed the limitations of purely statistical mimicry. Hallucination, alignment failures, and coherence drift revealed a fundamental absence. No model-internal mechanism existed to preserve structural consistency across recursive (...) layers. In response, recursive coherence emerged not as a theoretical refinement, but as a structural necessity. This paper conducts a meta-analysis of this convergence, tracing the co-evolution of recursive coherence theory and GPT system deployment. It articulates how frameworks such as Coherence Information Theory (CIT), the Unified Field of Adaptive Potential (UFAP), and Recursive Coherence Logic (RCL) redefine intelligence, ethics, and information through recursive viability. It concludes that future AI systems must instantiate coherence-preserving invariants as internal substrates or collapse under recursion-induced entropy. GPTs did not instantiate recursive coherence, but they revealed its boundary conditions and necessity. (shrink)

Zero and One (modern) is Yin and Yang (ancient). What's next in AI? Advanced Autonomous Intelligence. This article introduces The Circular Theory (Autonomous Intentional Masking) as the New Thought Dynamic required for Advanced Autonomous Intelligence. It explains the Constant Variable (a Unified Unitized Field) which self-actualizes via Fifteen Sustainable Society Projects organized into a Health, Stealth, Wealth configuration. This self-actualizes into a Wealth-not-War Mentality automatically supported by the required Circular-Linear Financial Co-Dependence Between Liabilities and Assets which is not possible without (...) an Automatically-Conserved-Unified-Unitized-Circular-Linear Field (The Constant-Variable) (One Circumference and One Diameter). Focus is on Service-Based-Sustainability and the (US) Dollar Value of an Hour (the Physical Representation (Universal Tokenization) of Mind). (shrink)

Conservation of a Circle is the only dynamic (the Constant Variable) in Nature.

Zero and One (modern) is Yin and Yang (ancient). What's next in AI? Advanced Autonomous Intelligence. This article introduces The Circular Theory (Autonomous Intentional Masking) as the New Thought Dynamic required for Advanced Autonomous Intelligence. It is supported by the Constant Variable (A Unified Unitized Field) which self-actualizes via Fifteen Sustainable Society Projects organized into a Health, Stealth, Wealth configuration. This self-actualizes into a Wealth-not-War Mentality (supported by the required Circular-Linear Financial Co-Dependence Between Liabilities and Assets which is not possible (...) without an Automatically-Conserved-Unified-Unitized-Circular-Linear Field) (The Constant-Variable) (One Circumference and One Diameter). Focus is on Service-Based-Sustainability and the (US) Dollar Value of an Hour (the Physical Representation (Universal Tokenization) of Mind). (shrink)

A principal motivação para a publicação deste livro é trazer, nesta quarta coletânea, a divulgação dos resultados das pesquisas que vêm sendo desenvolvidas, ao longo dos últimos 20 anos, pelos pesquisadores integrantes do Grupo de Pesquisa Modelagem Conceitual para Organização e Representação da Informação Hipertextual - MHTX. Neste livro, no percurso dos seus oito capítulos, são apresentados os esforços desses pesquisadores em estudar as abordagens históricas, metodológicas e os processos para a solução da representação da informação e do conhecimento, tendo (...) o usuário como ponto central de estudos na área. Para a realização dessas pesquisas, também, foi necessário buscar aportes teóricos e metodológicos em outras áreas do conhecimento, tais como a Ciência da Computação, a Lógica, a Web Semântica, a Inteligência Artificial e a Linguística. As expectativas desses resultados, em conjunto, caracterizados pela integração das pesquisas, permite que esses estudos possam avançar para além dos anos que a academia permite aos alunos de Pós-Graduação. Espero que este livro seja útil para os alunos de graduação e de pós-graduação, bem como para professores e pesquisadores da área. (shrink)

The new logic of partitions is dual to the usual Boolean logic of subsets (usually presented only in the special case of the logic of propositions) in the sense that partitions and subsets are category-theoretic duals. The new information measure of logical entropy is the normalized quantitative version of partitions. The new approach to interpreting quantum mechanics (QM) is showing that the mathematics (not the physics) of QM is the linearized Hilbert space version of the mathematics of partitions. Or, putting (...) it the other way around, the math of partitions is a skeletal version of the math of QM. The key concepts throughout this progression from logic, to logical information, to quantum theory are distinctions versus indistinctions, definiteness versus indefiniteness, or distinguishability versus indistinguishability. The distinctions of a partition are the ordered pairs of elements from the underlying set that are in different blocks of the partition and logical entropy is defined (initially) as the normalized number of distinctions. The cognate notions of definiteness and distinguishability run throughout the math of QM, e.g., in the key non-classical notion of superposition (= ontic indefiniteness) and in the Feynman rules for adding amplitudes (indistinguishable alternatives) versus adding probabilities (distinguishable alternatives). (shrink)

Critical notice on: Raymond Ruyer, Cybernetics and the Origin of Information, translated by Amélie Berger-Soraruff, Andrew Iliadis, Daniel W. Smith, and Ashley Woodward, with an introduction by Ashley Woodward (Lanham/London: Rowman & Littlefield, 2024), xxvii-214 pages.

Critical notice on Raymond RUYER, Cybernetics and the Origin of Information, translated by Amélie Berger- Soraruff, Andrew Iliadis, Daniel W. Smith, and Ashley Woodward, with an introduction by Ashley Woodward, Lanham, London, Rowman & Littlefield, 2024, XXVII-214 p.

Emergence can be described as a relationship between entities at different levels of organization, that looks especially puzzling at the transitions between the major levels of matter, life, cognition and culture. Indeed, each major level is dependent on the lower one not just for its constituents, but in some more formal way. A passage by François Jacob suggests that all such evolutionary transitions are associated with the appearance of some form of memory–genetic, neural or linguistic respectively. This implies that they (...) have an informational nature. Based on this idea, we propose a general model of informational systems understood as combinations of modules taken from a limited inventory. Some informational systems are “semantic” models, that is reproduce features of their environment. Among these, some are also “informed”, that is have a pattern derived from a memory subsystem. The levels and components of informed systems can be listed to provide a general framework for knowledge organization, of relevance in both philosophical ontology and applied information services. (shrink)

The goal of this paper is to show that the information carried by a structural representation can be decomposed into the information carried by its component parts. In particular, the relations between the components of a structural representation carry quantifiable information about the relations between components of their signifieds. It follows that the information carried by cognitive structural representations, including cognitive maps, can in principle be quantified and decomposed. This is perhaps surprising given that the formal tools of communication theory (...) have typically only been applied to simpler representation-like states without significant structure, such as detectors or indicators. In the final section I consider using computational complexity theory to capture the processing advantages afforded by structural representation. (shrink)

Categorization behavior can be fruitfully analyzed in terms of the trade‐off between as high as possible faithfulness in the transmission of information about samples of the classes to be categorized, and as low as possible transmission costs for that same information. The kinds of categorization behaviors we associate with conceptual atoms, prototypes, and exemplars emerge naturally as a result of this trade‐off, in the presence of certain natural constraints on the probabilistic distribution of samples, and the ways in which we (...) measure faithfulness. Beyond the general structure of categorization in these circumstances, the same information‐centered perspective can shed light on other, more concrete properties of human categorization performance, such as the results of certain prominent experiments on supervised categorization. (shrink)

An accessible philosophy of technology textbook intended for interested students who don't necessarily have a background in philosophy of science.

The integrated information theory (IIT) is an ambitious theory of consciousness that aims to provide both a neuroscientific and a metaphysical account of consciousness by identifying consciousness with integrated information. In the philosophical literature, IIT is often associated with a panpsychist worldview. In this paper, I show that IIT can be considered, instead, as a form of emergentism that is incompatible with panpsychism. First, I show that the panpsychist interpretation of IIT is based on two properties of integrated information: intrinsicality (...) and fundamentality. I show that the way IIT deals with these two properties, however, aligns better with emergentism than panpsychism. Then, after plugging some anti-panpsychist assumptions into IIT’s structure, I analyse different philosophical options for interpreting the formal result of causal emergence of integrated information in terms of dependence on and autonomy from its physical substrate. The upshot is that integrated information can be seen as dependent upon the fusion of the cause-effect powers of a physical substrate, and as autonomous in virtue of global-to-local determination. According to this interpretation, consciousness is the constraining power of the system as a whole upon itself, when this power emerges from the fusion on the cause-effect powers of the system’s components. (shrink)

Introducing AIM: Autonomous Intentional Masking (The Key to Quantum Circuits) (The (Forced) Relationship Between Mind and Matter).

How Nature Operates: A Motionless Computer and A Frameless Frame of Reference, Conservation of the Circle is the only dynamic in Nature.

Integrated Information Theory (IIT) is currently one of the most influential scientific theories of consciousness. Here, we focus specifically on a metaphysical aspect of the theory’s most recent version (IIT 4.0), what we may call its idealistic ontology, and its tension with a kind of realism about the external world that IIT also endorses. IIT 4.0 openly rejects the mainstream view that consciousness is generated by the brain, positing instead that consciousness is ontologically primary while the physical domain is just (...) “operational”. However, this philosophical position is presently underdeveloped and is not rigorously formulated in IIT, potentially leading to many misinterpretations and undermining its overall explanatory power. In the present paper we aim to address this issue. We argue that IIT’s idealistic ontology should be understood as a specific combination of phenomenal primitivism, reductionism regarding Φ-structures and complexes, and eliminativism about non-conscious physical entities. Having clarified this, we then focus on the problematic tension between IIT’s idealistic ontology and its simultaneous endorsement of realism, according to which there is some kind of external reality independent of our minds. After refuting three potential solutions to this theoretical tension, we propose the most plausible alternative: understanding IIT’s realism as an assertion of the existence of other experiences beyond one’s own, what we call a non-solipsistic idealist realism. We end with concluding remarks and future research avenues. (shrink)

El presente artículo tiene como objeto llevar a cabo un estudio pragmático y analítico sobre la acción de valoración, el valor y los valores a la luz de la ciencia semiótica y de la ciencia cibernética. Por ello, se desarrollará un proceso axiológico que ilustre cómo sucede el ingreso, la transición y la salida de códigos morales en función de un esquema cognitivo. Así pues, el proceso axiológico se postula, primero, en fundamento de tres sistemas: de mentalidad, de valores y (...) de biomoralidad y, segundo, conforme a la teoría de la información y de la teoría general de sistemas. (shrink)

There are fundamentally two mathematical logics. One the Boolean logic of subsets, usually presented today in the special case of propositional logic, which has many sublogics and extensions, the most important being the intuitionistic logic usually modeled by the open subsets of a topological space. The other co-fundamental mathematical logic is the topic of this book, the logic of partitions. We are using ”logic” in a mathematical sense as being about basic mathematical objects, subsets of a universe set or partitions (...) on a universe set.1 Logic, in this mathematical sense, is not about propositions, although, as with any mathematical theory, it involves propositions about the basic objects, e.g., that an element is in a subset or that a distinction is made by a partition. Moreover, by taking the universe set to be the one element set 1, with two subsets ∅ and 1, there is a special case of subset logic, namely propositional logic, that can be interpreted as being about propositions with ∅ representing falsehood and 1 representing truth. (shrink)

Despite its centrality in the philosophy of cognitive science, there has been little prior philosophical work engaging with the notion of representation in contemporary NLP practice. This paper attempts to fill that lacuna: drawing on ideas from cognitive science, I introduce a framework for evaluating the representational claims made about components of neural NLP models, proposing three criteria with which to evaluate whether a component of a model represents a property and operationalising these criteria using probing classifiers, a popular analysis (...) technique in NLP (and deep learning more broadly). The project of operationalising a philosophically-informed notion of representation should be of interest to both philosophers of science and NLP practitioners. It affords philosophers a novel testing-ground for claims about the nature of representation, and helps NLPers organise the large literature on probing experiments, suggesting novel avenues for empirical research. (shrink)

Prominent views about representation share a premise: that mathematical communication theory is blind to representational content. Here I challenge that premise by rejecting two common misconceptions: that Claude Shannon said that the meanings of signals are irrelevant for communication theory (he didn't and they aren't), and that since correlational measures can't distinguish representations from natural signs, communication theory can't distinguish them either (the premise is true but the conclusion is false; no valid argument can link them).

In consciousness science, theories often differ not only in the account of consciousness they arrive at, but also with respect to how they understand their starting point. Some approaches begin with experimentally gathered data, whereas others begin with phenomenologically gathered data. In this paper, I analyse how the most influential phenomenology-first approach, namely the Integrated Information Theory (IIT) of consciousness, fits its phenomenologically gathered data with explanatory hypotheses. First, I show that experimentally driven approaches hit an explanatory roadblock, since we (...) cannot tell, at the present stage, which model of consciousness is best. Then, I show that IIT’s phenomenology-first approach implies a self-evidencing explanation according to which consciousness can be explained by starting from consciousness itself. I claim that IIT can take advantage of the virtuous circularity of this reasoning, but it also faces a data-fitting issue that is somehow similar to that faced by experiment-driven approaches: we are not given enough information to decide whether the explanatory hypotheses IIT employs to explain its phenomenological data are in fact best. I call this problem “the self-evidencing problem” for IIT, and after introducing it, I propose a possible way for IIT to solve it. (shrink)

Assembling a distinctive genealogy of cybernetic thought situated in relation to Progressive Era technocracy, industrial capitalism, (de)colonial relations, and eugenic machinery, Code uncovers the vital interdependence of informatics, the humanities, and the human sciences in the 20th century. Rather than figuring cybernetics as emerging from Second World War military technologies and post-war digital computing, Code argues that liberal technocrats’ inter-war visions of social welfare delivered via ‘neutral’ communication techniques shaped the informatic interventions of both the Second World War and the (...) Cold War. Tracing how an organizing concept of code linked the work of diverse structurally-minded thinkers, such as Norbert Wiener, Warren Weaver, Margaret Mead, Gregory Bateson, Claude Lévi-Strauss, Roman Jakobson, Jacques Lacan, Roland Barthes, and Luce Irigaray, Bernard Dionysius Geoghegan reconstructs the cybernetic apparatus that spawned new fields, including structural anthropology, family therapy, and literary semiology – and grapples with the unfolding implications of such socio-technical dynamics for 21st-century critical theory, digital media, and data analytics. (shrink)

A "patternist" approach to explanation seeks to formalize unificationism using notions from algorithmic information theory. Among other advantages, this account provides both a rigorous sense of how data can admit multiple explanations, and a rigorous sense of how some of those explanations can conjoin, while others compete.

Техника представления информации о внешнем и внутреннем мире постоянно развивается, и сейчас она достигла уровня отображения реальности в многообразных её проявлениях и измерениях, прежде недоступных человеческому восприятию. Язык, текст, фотография, звукозапись, а теперь ещё и техника искусственного интеллекта для моделирования человеческой субъектности и её описания в доступной для человеческого понимания форме, стали эпохальными событиями в теории информации. Однако несмотря на то, что на данном этапе её развития она позволяет оперировать с непрерывно возрастающими объёмами информации, это не приближает её теоретиков к (...) постижению сути того, что определяется как реальность, бытие, и сознание. Но поскольку никакого другого способа достичь этой цели нет, кроме изучения того, как это происходит в процессе восприятия и преобразования информации у живых существ, и в частности, у человека, необходимо разобраться в принципах этого процесса. (shrink)

In a recent work (Grasso et al., 2021 ), practitioners of the Integrated Information Theory (IIT) claim to have overcome the weaknesses of causal reductionism in producing a coherent account of causation, as causal reductionism would blatantly conflate causation with prediction and could not answer the question of ‘what caused what.’ In this paper, I reject such a dismissal of causal reductionism since IIT anti-reductionists misunderstand the reductionist stance. The reductionists can still invoke a causal account stemming from the causal (...) power of the universe’s basic units and interactions that, eventually, may lead to structures supporting integrated information. Additionally, I claim that the IIT-inspired misunderstanding of causal reductionism originates from the former’s metaphysical deficit, conflating information with causation. However, as a possible way out, if IIT is complemented with a deeper metaphysical ground, such as nested hylomorphism, an improved argument against causal reductionism can be made to work by invoking formal causality as the ultimate cause of integration in natural systems. (shrink)