Mailman 3 February 2024 - Python-checkins

[3.11] gh-87115: Set `__main__.__spec__` to `None` in pdb (GH-116141) (#116155)
by iritkatriel Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/d3282f541182c80b9e4df4300b5ed687c7… commit: d3282f541182c80b9e4df4300b5ed687c718c14b branch: 3.11 author: Tian Gao <gaogaotiantian(a)hotmail.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2024-02-29T23:24:32Z summary: [3.11] gh-87115: Set `__main__.__spec__` to `None` in pdb (GH-116141) (#116155) * gh-87115: Set `__main__.__spec__` to `None` in pdb (#116141) (cherry picked from commit ccfc042bbf31e53c44b8aae444afd8365b798422) * [3.11] gh-87115: Set `__main__.__spec__` to `None` in pdb (GH-116141) (cherry picked from commit ccfc042bbf31e53c44b8aae444afd8365b798422) Co-authored-by: Tian Gao <gaogaotiantian(a)hotmail.com> files: A Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst M Lib/pdb.py M Lib/test/test_pdb.py diff --git a/Lib/pdb.py b/Lib/pdb.py index 4a4a0b9d90f6d2..fb7b0eccd16d1c 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -153,6 +153,7 @@ def namespace(self): __name__='__main__', __file__=self, __builtins__=__builtins__, + __spec__=None, ) @property diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index eed02737e8bd48..15d9eaa1f93cb2 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -1862,6 +1862,18 @@ def bœr(): ('bœr', 1), ) + def test_spec(self): + # Test that __main__.__spec__ is set to None when running a script + script = """ + import __main__ + print(__main__.__spec__) + """ + + commands = "continue" + + stdout, _ = self.run_pdb_script(script, commands) + self.assertIn('None', stdout) + def test_issue7964(self): # open the file as binary so we can force \r\n newline with open(os_helper.TESTFN, 'wb') as f: diff --git a/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst b/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst new file mode 100644 index 00000000000000..844340583cd456 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst @@ -0,0 +1 @@ +Set ``__main__.__spec__`` to ``None`` when running a script with :mod:`pdb`

1 0

[3.12] gh-87115: Set `__main__.__spec__` to `None` in pdb (GH-116141) (#116154)
by iritkatriel Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/40e9295a581f2908f2cdf09b1726822379… commit: 40e9295a581f2908f2cdf09b1726822379ea7bd0 branch: 3.12 author: Tian Gao <gaogaotiantian(a)hotmail.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2024-02-29T23:24:09Z summary: [3.12] gh-87115: Set `__main__.__spec__` to `None` in pdb (GH-116141) (#116154) * gh-87115: Set `__main__.__spec__` to `None` in pdb (#116141) (cherry picked from commit ccfc042bbf31e53c44b8aae444afd8365b798422) * [3.12] gh-87115: Set `__main__.__spec__` to `None` in pdb (GH-116141) (cherry picked from commit ccfc042bbf31e53c44b8aae444afd8365b798422) Co-authored-by: Tian Gao <gaogaotiantian(a)hotmail.com> files: A Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst M Lib/pdb.py M Lib/test/test_pdb.py diff --git a/Lib/pdb.py b/Lib/pdb.py index a838a26b038df6..494e640cda2757 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -154,6 +154,7 @@ def namespace(self): __name__='__main__', __file__=self, __builtins__=__builtins__, + __spec__=None, ) @property diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index 51b844262ed1e8..0e1b1c9864f8c1 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -2013,6 +2013,18 @@ def bœr(): ('bœr', 1), ) + def test_spec(self): + # Test that __main__.__spec__ is set to None when running a script + script = """ + import __main__ + print(__main__.__spec__) + """ + + commands = "continue" + + stdout, _ = self.run_pdb_script(script, commands) + self.assertIn('None', stdout) + def test_issue7964(self): # open the file as binary so we can force \r\n newline with open(os_helper.TESTFN, 'wb') as f: diff --git a/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst b/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst new file mode 100644 index 00000000000000..844340583cd456 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst @@ -0,0 +1 @@ +Set ``__main__.__spec__`` to ``None`` when running a script with :mod:`pdb`

1 0

gh-103092: Add a mutex to make the PRNG state of rotatingtree concurrent-safe (#115301)
by erlend-aasland Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/ca56c3a172e4ca08c278ddaf66ac2943a1… commit: ca56c3a172e4ca08c278ddaf66ac2943a1d93288 branch: main author: AN Long <aisk(a)users.noreply.github.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2024-03-01T00:04:16+01:00 summary: gh-103092: Add a mutex to make the PRNG state of rotatingtree concurrent-safe (#115301) files: A Misc/NEWS.d/next/Library/2024-02-12-11-42-48.gh-issue-103092.sGMKr0.rst M Modules/_lsprof.c M Modules/rotatingtree.c M Tools/c-analyzer/cpython/globals-to-fix.tsv diff --git a/Misc/NEWS.d/next/Library/2024-02-12-11-42-48.gh-issue-103092.sGMKr0.rst b/Misc/NEWS.d/next/Library/2024-02-12-11-42-48.gh-issue-103092.sGMKr0.rst new file mode 100644 index 00000000000000..47701396c81737 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-02-12-11-42-48.gh-issue-103092.sGMKr0.rst @@ -0,0 +1 @@ +Isolate :mod:`_lsprof` (apply :pep:`687`). diff --git a/Modules/_lsprof.c b/Modules/_lsprof.c index f1cee7cb6f66bf..a76c3dea555783 100644 --- a/Modules/_lsprof.c +++ b/Modules/_lsprof.c @@ -1005,9 +1005,7 @@ _lsprof_exec(PyObject *module) static PyModuleDef_Slot _lsprofslots[] = { {Py_mod_exec, _lsprof_exec}, - // XXX gh-103092: fix isolation. - {Py_mod_multiple_interpreters, Py_MOD_MULTIPLE_INTERPRETERS_NOT_SUPPORTED}, - //{Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED}, + {Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED}, {0, NULL} }; diff --git a/Modules/rotatingtree.c b/Modules/rotatingtree.c index 07e08bc3167c0a..217e495b3d2a9d 100644 --- a/Modules/rotatingtree.c +++ b/Modules/rotatingtree.c @@ -1,3 +1,9 @@ +#ifndef Py_BUILD_CORE_BUILTIN +# define Py_BUILD_CORE_MODULE 1 +#endif + +#include "Python.h" +#include "pycore_lock.h" #include "rotatingtree.h" #define KEY_LOWER_THAN(key1, key2) ((char*)(key1) < (char*)(key2)) @@ -10,17 +16,20 @@ static unsigned int random_value = 1; static unsigned int random_stream = 0; +static PyMutex random_mutex = {0}; static int randombits(int bits) { int result; + PyMutex_Lock(&random_mutex); if (random_stream < (1U << bits)) { random_value *= 1082527; random_stream = random_value; } result = random_stream & ((1<<bits)-1); random_stream >>= bits; + PyMutex_Unlock(&random_mutex); return result; } diff --git a/Tools/c-analyzer/cpython/globals-to-fix.tsv b/Tools/c-analyzer/cpython/globals-to-fix.tsv index 45119664af4362..686a3d3160cc90 100644 --- a/Tools/c-analyzer/cpython/globals-to-fix.tsv +++ b/Tools/c-analyzer/cpython/globals-to-fix.tsv @@ -482,3 +482,4 @@ Modules/readline.c - sigwinch_ohandler - Modules/readline.c - completed_input_string - Modules/rotatingtree.c - random_stream - Modules/rotatingtree.c - random_value - +Modules/rotatingtree.c - random_mutex -

1 0

gh-115398: Expose Expat >=2.6.0 reparse deferral API (CVE-2023-52425) (GH-115623)
by gpshead Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/6a95676bb526261434dd068d6c49927c44… commit: 6a95676bb526261434dd068d6c49927c44d24a9b branch: main author: Sebastian Pipping <sebastian(a)pipping.org> committer: gpshead <greg(a)krypto.org> date: 2024-02-29T14:52:50-08:00 summary: gh-115398: Expose Expat >=2.6.0 reparse deferral API (CVE-2023-52425) (GH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 . ### Notes - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <snild(a)sony.com> and by core dev Serhiy Storchaka. files: A Misc/NEWS.d/next/Security/2024-02-18-03-14-40.gh-issue-115398.tzvxH8.rst M Doc/library/pyexpat.rst M Doc/library/xml.etree.elementtree.rst M Doc/whatsnew/3.13.rst M Include/pyexpat.h M Lib/test/test_pyexpat.py M Lib/test/test_sax.py M Lib/test/test_xml_etree.py M Lib/xml/etree/ElementTree.py M Lib/xml/sax/expatreader.py M Misc/sbom.spdx.json M Modules/_elementtree.c M Modules/clinic/_elementtree.c.h M Modules/clinic/pyexpat.c.h M Modules/expat/pyexpatns.h M Modules/pyexpat.c diff --git a/Doc/library/pyexpat.rst b/Doc/library/pyexpat.rst index a6ae8fdaa4991c..c897ec9e47b7ca 100644 --- a/Doc/library/pyexpat.rst +++ b/Doc/library/pyexpat.rst @@ -196,6 +196,37 @@ XMLParser Objects :exc:`ExpatError` to be raised with the :attr:`code` attribute set to ``errors.codes[errors.XML_ERROR_CANT_CHANGE_FEATURE_ONCE_PARSING]``. +.. method:: xmlparser.SetReparseDeferralEnabled(enabled) + + .. warning:: + + Calling ``SetReparseDeferralEnabled(False)`` has security implications, + as detailed below; please make sure to understand these consequences + prior to using the ``SetReparseDeferralEnabled`` method. + + Expat 2.6.0 introduced a security mechanism called "reparse deferral" + where instead of causing denial of service through quadratic runtime + from reparsing large tokens, reparsing of unfinished tokens is now delayed + by default until a sufficient amount of input is reached. + Due to this delay, registered handlers may — depending of the sizing of + input chunks pushed to Expat — no longer be called right after pushing new + input to the parser. Where immediate feedback and taking over responsiblity + of protecting against denial of service from large tokens are both wanted, + calling ``SetReparseDeferralEnabled(False)`` disables reparse deferral + for the current Expat parser instance, temporarily or altogether. + Calling ``SetReparseDeferralEnabled(True)`` allows re-enabling reparse + deferral. + + .. versionadded:: 3.13 + +.. method:: xmlparser.GetReparseDeferralEnabled() + + Returns whether reparse deferral is currently enabled for the given + Expat parser instance. + + .. versionadded:: 3.13 + + :class:`xmlparser` objects have the following attributes: diff --git a/Doc/library/xml.etree.elementtree.rst b/Doc/library/xml.etree.elementtree.rst index 75a7915c15240d..19c7af452e2b71 100644 --- a/Doc/library/xml.etree.elementtree.rst +++ b/Doc/library/xml.etree.elementtree.rst @@ -166,6 +166,11 @@ data but would still like to have incremental parsing capabilities, take a look at :func:`iterparse`. It can be useful when you're reading a large XML document and don't want to hold it wholly in memory. +Where *immediate* feedback through events is wanted, calling method +:meth:`XMLPullParser.flush` can help reduce delay; +please make sure to study the related security notes. + + Finding interesting elements ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -1387,6 +1392,19 @@ XMLParser Objects Feeds data to the parser. *data* is encoded data. + + .. method:: flush() + + Triggers parsing of any previously fed unparsed data, which can be + used to ensure more immediate feedback, in particular with Expat >=2.6.0. + The implementation of :meth:`flush` temporarily disables reparse deferral + with Expat (if currently enabled) and triggers a reparse. + Disabling reparse deferral has security consequences; please see + :meth:`xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` for details. + + .. versionadded:: 3.13 + + :meth:`XMLParser.feed` calls *target*\'s ``start(tag, attrs_dict)`` method for each opening tag, its ``end(tag)`` method for each closing tag, and data is processed by method ``data(data)``. For further supported callback @@ -1448,6 +1466,17 @@ XMLPullParser Objects Feed the given bytes data to the parser. + .. method:: flush() + + Triggers parsing of any previously fed unparsed data, which can be + used to ensure more immediate feedback, in particular with Expat >=2.6.0. + The implementation of :meth:`flush` temporarily disables reparse deferral + with Expat (if currently enabled) and triggers a reparse. + Disabling reparse deferral has security consequences; please see + :meth:`xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` for details. + + .. versionadded:: 3.13 + .. method:: close() Signal the parser that the data stream is terminated. Unlike diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 3a277d7ce1585f..d08c63e7b2c2c5 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -174,6 +174,17 @@ Other Language Changes (Contributed by Victor Stinner in :gh:`114570`.) +* Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) + by adding five new methods: + + * :meth:`xml.etree.ElementTree.XMLParser.flush` + * :meth:`xml.etree.ElementTree.XMLPullParser.flush` + * :meth:`xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` + * :meth:`xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` + * :meth:`!xml.sax.expatreader.ExpatParser.flush` + + (Contributed by Sebastian Pipping in :gh:`115623`.) + New Modules =========== diff --git a/Include/pyexpat.h b/Include/pyexpat.h index 07020b5dc964cb..9824d099c3df7d 100644 --- a/Include/pyexpat.h +++ b/Include/pyexpat.h @@ -48,8 +48,10 @@ struct PyExpat_CAPI enum XML_Status (*SetEncoding)(XML_Parser parser, const XML_Char *encoding); int (*DefaultUnknownEncodingHandler)( void *encodingHandlerData, const XML_Char *name, XML_Encoding *info); - /* might be none for expat < 2.1.0 */ + /* might be NULL for expat < 2.1.0 */ int (*SetHashSalt)(XML_Parser parser, unsigned long hash_salt); + /* might be NULL for expat < 2.6.0 */ + XML_Bool (*SetReparseDeferralEnabled)(XML_Parser parser, XML_Bool enabled); /* always add new stuff to the end! */ }; diff --git a/Lib/test/test_pyexpat.py b/Lib/test/test_pyexpat.py index d941a1a8f9ebc6..1d56ccd71cf962 100644 --- a/Lib/test/test_pyexpat.py +++ b/Lib/test/test_pyexpat.py @@ -755,5 +755,59 @@ def resolve_entity(context, base, system_id, public_id): self.assertEqual(handler_call_args, [("bar", "baz")]) +class ReparseDeferralTest(unittest.TestCase): + def test_getter_setter_round_trip(self): + parser = expat.ParserCreate() + enabled = (expat.version_info >= (2, 6, 0)) + + self.assertIs(parser.GetReparseDeferralEnabled(), enabled) + parser.SetReparseDeferralEnabled(False) + self.assertIs(parser.GetReparseDeferralEnabled(), False) + parser.SetReparseDeferralEnabled(True) + self.assertIs(parser.GetReparseDeferralEnabled(), enabled) + + def test_reparse_deferral_enabled(self): + if expat.version_info < (2, 6, 0): + self.skipTest(f'Expat {expat.version_info} does not ' + 'support reparse deferral') + + started = [] + + def start_element(name, _): + started.append(name) + + parser = expat.ParserCreate() + parser.StartElementHandler = start_element + self.assertTrue(parser.GetReparseDeferralEnabled()) + + for chunk in (b'<doc', b'/>'): + parser.Parse(chunk, False) + + # The key test: Have handlers already fired? Expecting: no. + self.assertEqual(started, []) + + parser.Parse(b'', True) + + self.assertEqual(started, ['doc']) + + def test_reparse_deferral_disabled(self): + started = [] + + def start_element(name, _): + started.append(name) + + parser = expat.ParserCreate() + parser.StartElementHandler = start_element + if expat.version_info >= (2, 6, 0): + parser.SetReparseDeferralEnabled(False) + self.assertFalse(parser.GetReparseDeferralEnabled()) + + for chunk in (b'<doc', b'/>'): + parser.Parse(chunk, False) + + # The key test: Have handlers already fired? Expecting: yes. + self.assertEqual(started, ['doc']) + + if __name__ == "__main__": unittest.main() diff --git a/Lib/test/test_sax.py b/Lib/test/test_sax.py index eda4e6a46df437..97e96668f85c8a 100644 --- a/Lib/test/test_sax.py +++ b/Lib/test/test_sax.py @@ -19,6 +19,7 @@ from io import BytesIO, StringIO import codecs import os.path +import pyexpat import shutil import sys from urllib.error import URLError @@ -1214,6 +1215,56 @@ def test_expat_incremental_reset(self): self.assertEqual(result.getvalue(), start + b"<doc>text</doc>") + def test_flush_reparse_deferral_enabled(self): + if pyexpat.version_info < (2, 6, 0): + self.skipTest(f'Expat {pyexpat.version_info} does not support reparse deferral') + + result = BytesIO() + xmlgen = XMLGenerator(result) + parser = create_parser() + parser.setContentHandler(xmlgen) + + for chunk in ("<doc", ">"): + parser.feed(chunk) + + self.assertEqual(result.getvalue(), start) # i.e. no elements started + self.assertTrue(parser._parser.GetReparseDeferralEnabled()) + + parser.flush() + + self.assertTrue(parser._parser.GetReparseDeferralEnabled()) + self.assertEqual(result.getvalue(), start + b"<doc>") + + parser.feed("</doc>") + parser.close() + + self.assertEqual(result.getvalue(), start + b"<doc></doc>") + + def test_flush_reparse_deferral_disabled(self): + result = BytesIO() + xmlgen = XMLGenerator(result) + parser = create_parser() + parser.setContentHandler(xmlgen) + + for chunk in ("<doc", ">"): + parser.feed(chunk) + + if pyexpat.version_info >= (2, 6, 0): + parser._parser.SetReparseDeferralEnabled(False) + + self.assertEqual(result.getvalue(), start) # i.e. no elements started + self.assertFalse(parser._parser.GetReparseDeferralEnabled()) + + parser.flush() + + self.assertFalse(parser._parser.GetReparseDeferralEnabled()) + self.assertEqual(result.getvalue(), start + b"<doc>") + + parser.feed("</doc>") + parser.close() + + self.assertEqual(result.getvalue(), start + b"<doc></doc>") + # ===== Locator support def test_expat_locator_noinfo(self): diff --git a/Lib/test/test_xml_etree.py b/Lib/test/test_xml_etree.py index c535d631bb646f..14df482ba6c207 100644 --- a/Lib/test/test_xml_etree.py +++ b/Lib/test/test_xml_etree.py @@ -121,10 +121,6 @@ </foo> """ -fails_with_expat_2_6_0 = (unittest.expectedFailure - if pyexpat.version_info >= (2, 6, 0) else - lambda test: test) - def checkwarnings(*filters, quiet=False): def decorator(test): def newtest(*args, **kwargs): @@ -1462,12 +1458,14 @@ def test_attlist_default(self): class XMLPullParserTest(unittest.TestCase): - def _feed(self, parser, data, chunk_size=None): + def _feed(self, parser, data, chunk_size=None, flush=False): if chunk_size is None: parser.feed(data) else: for i in range(0, len(data), chunk_size): parser.feed(data[i:i+chunk_size]) + if flush: + parser.flush() def assert_events(self, parser, expected, max_events=None): self.assertEqual( @@ -1485,34 +1483,32 @@ def assert_event_tags(self, parser, expected, max_events=None): self.assertEqual([(action, elem.tag) for action, elem in events], expected) - def test_simple_xml(self, chunk_size=None): + def test_simple_xml(self, chunk_size=None, flush=False): parser = ET.XMLPullParser() self.assert_event_tags(parser, []) - self._feed(parser, "\n", chunk_size) + self._feed(parser, "\n", chunk_size, flush) self.assert_event_tags(parser, []) self._feed(parser, "<root>\n <element key='value'>text</element", - chunk_size) + chunk_size, flush) self.assert_event_tags(parser, []) - self._feed(parser, ">\n", chunk_size) + self._feed(parser, ">\n", chunk_size, flush) self.assert_event_tags(parser, [('end', 'element')]) - self._feed(parser, "<element>text</element>tail\n", chunk_size) - self._feed(parser, "<empty-element/>\n", chunk_size) + self._feed(parser, "<element>text</element>tail\n", chunk_size, flush) + self._feed(parser, "<empty-element/>\n", chunk_size, flush) self.assert_event_tags(parser, [ ('end', 'element'), ('end', 'empty-element'), ]) - self._feed(parser, "</root>\n", chunk_size) + self._feed(parser, "</root>\n", chunk_size, flush) self.assert_event_tags(parser, [('end', 'root')]) self.assertIsNone(parser.close()) - @fails_with_expat_2_6_0 def test_simple_xml_chunk_1(self): - self.test_simple_xml(chunk_size=1) + self.test_simple_xml(chunk_size=1, flush=True) - @fails_with_expat_2_6_0 def test_simple_xml_chunk_5(self): - self.test_simple_xml(chunk_size=5) + self.test_simple_xml(chunk_size=5, flush=True) def test_simple_xml_chunk_22(self): self.test_simple_xml(chunk_size=22) @@ -1711,6 +1707,57 @@ def test_unknown_event(self): with self.assertRaises(ValueError): ET.XMLPullParser(events=('start', 'end', 'bogus')) + def test_flush_reparse_deferral_enabled(self): + if pyexpat.version_info < (2, 6, 0): + self.skipTest(f'Expat {pyexpat.version_info} does not ' + 'support reparse deferral') + + parser = ET.XMLPullParser(events=('start', 'end')) + + for chunk in ("<doc", ">"): + parser.feed(chunk) + + self.assert_event_tags(parser, []) # i.e. no elements started + if ET is pyET: + self.assertTrue(parser._parser._parser.GetReparseDeferralEnabled()) + + parser.flush() + + self.assert_event_tags(parser, [('start', 'doc')]) + if ET is pyET: + self.assertTrue(parser._parser._parser.GetReparseDeferralEnabled()) + + parser.feed("</doc>") + parser.close() + + self.assert_event_tags(parser, [('end', 'doc')]) + + def test_flush_reparse_deferral_disabled(self): + parser = ET.XMLPullParser(events=('start', 'end')) + + for chunk in ("<doc", ">"): + parser.feed(chunk) + + if pyexpat.version_info >= (2, 6, 0): + if not ET is pyET: + self.skipTest(f'XMLParser.(Get|Set)ReparseDeferralEnabled ' + 'methods not available in C') + parser._parser._parser.SetReparseDeferralEnabled(False) + + self.assert_event_tags(parser, []) # i.e. no elements started + if ET is pyET: + self.assertFalse(parser._parser._parser.GetReparseDeferralEnabled()) + + parser.flush() + + self.assert_event_tags(parser, [('start', 'doc')]) + if ET is pyET: + self.assertFalse(parser._parser._parser.GetReparseDeferralEnabled()) + + parser.feed("</doc>") + parser.close() + + self.assert_event_tags(parser, [('end', 'doc')]) # # xinclude tests (samples from appendix C of the xinclude specification) diff --git a/Lib/xml/etree/ElementTree.py b/Lib/xml/etree/ElementTree.py index a37fead41b750e..9e15d34d22aa6c 100644 --- a/Lib/xml/etree/ElementTree.py +++ b/Lib/xml/etree/ElementTree.py @@ -1320,6 +1320,11 @@ def read_events(self): else: yield event + def flush(self): + if self._parser is None: + raise ValueError("flush() called after end of stream") + self._parser.flush() + def XML(text, parser=None): """Parse XML document from string constant. @@ -1726,6 +1731,15 @@ def close(self): del self.parser, self._parser del self.target, self._target + def flush(self): + was_enabled = self.parser.GetReparseDeferralEnabled() + try: + self.parser.SetReparseDeferralEnabled(False) + self.parser.Parse(b"", False) + except self._error as v: + self._raiseerror(v) + finally: + self.parser.SetReparseDeferralEnabled(was_enabled) # -------------------------------------------------------------------- # C14N 2.0 diff --git a/Lib/xml/sax/expatreader.py b/Lib/xml/sax/expatreader.py index b9ad52692db8dd..ba3c1e98517429 100644 --- a/Lib/xml/sax/expatreader.py +++ b/Lib/xml/sax/expatreader.py @@ -214,6 +214,20 @@ def feed(self, data, isFinal=False): # FIXME: when to invoke error()? self._err_handler.fatalError(exc) + def flush(self): + if self._parser is None: + return + + was_enabled = self._parser.GetReparseDeferralEnabled() + try: + self._parser.SetReparseDeferralEnabled(False) + self._parser.Parse(b"", False) + except expat.error as e: + exc = SAXParseException(expat.ErrorString(e.code), e, self) + self._err_handler.fatalError(exc) + finally: + self._parser.SetReparseDeferralEnabled(was_enabled) + def _close_source(self): source = self._source try: diff --git a/Misc/NEWS.d/next/Security/2024-02-18-03-14-40.gh-issue-115398.tzvxH8.rst b/Misc/NEWS.d/next/Security/2024-02-18-03-14-40.gh-issue-115398.tzvxH8.rst new file mode 100644 index 00000000000000..97b23936928d91 --- /dev/null +++ b/Misc/NEWS.d/next/Security/2024-02-18-03-14-40.gh-issue-115398.tzvxH8.rst @@ -0,0 +1,8 @@ +Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding +five new methods: + +* ``xml.etree.ElementTree.XMLParser.flush`` +* ``xml.etree.ElementTree.XMLPullParser.flush`` +* ``xml.parsers.expat.xmlparser.GetReparseDeferralEnabled`` +* ``xml.parsers.expat.xmlparser.SetReparseDeferralEnabled`` +* ``xml.sax.expatreader.ExpatParser.flush`` diff --git a/Misc/sbom.spdx.json b/Misc/sbom.spdx.json index e28eaea81d6aae..27e6742292ac6d 100644 --- a/Misc/sbom.spdx.json +++ b/Misc/sbom.spdx.json @@ -132,11 +132,11 @@ "checksums": [ { "algorithm": "SHA1", - "checksumValue": "baa44fe4581895d42e8d5e83d8ce6a69b1c34dbe" + "checksumValue": "f50c899172acd93fc539007bfb43315b83d407e4" }, { "algorithm": "SHA256", - "checksumValue": "33a7b9ac8bf4571e23272cdf644c6f9808bd44c66b149e3c41ab3870d1888609" + "checksumValue": "d571b8258cfaa067a20adef553e5fcedd6671ca4a8841483496de031bd904567" } ], "fileName": "Modules/expat/pyexpatns.h" diff --git a/Modules/_elementtree.c b/Modules/_elementtree.c index 54451081211654..edd2f88a4881c3 100644 --- a/Modules/_elementtree.c +++ b/Modules/_elementtree.c @@ -3894,6 +3894,40 @@ _elementtree_XMLParser_close_impl(XMLParserObject *self) } } +/*[clinic input] +_elementtree.XMLParser.flush + +[clinic start generated code]*/ + +static PyObject * +_elementtree_XMLParser_flush_impl(XMLParserObject *self) +/*[clinic end generated code: output=42fdb8795ca24509 input=effbecdb28715949]*/ +{ + if (!_check_xmlparser(self)) { + return NULL; + } + + elementtreestate *st = self->state; + + if (EXPAT(st, SetReparseDeferralEnabled) == NULL) { + Py_RETURN_NONE; + } + + // NOTE: The Expat parser in the C implementation of ElementTree is not + // exposed to the outside; as a result we known that reparse deferral + // is currently enabled, or we would not even have access to function + // XML_SetReparseDeferralEnabled in the first place (which we checked + // for, a few lines up). + + EXPAT(st, SetReparseDeferralEnabled)(self->parser, XML_FALSE); + + PyObject *res = expat_parse(st, self, "", 0, XML_FALSE); + + EXPAT(st, SetReparseDeferralEnabled)(self->parser, XML_TRUE); + + return res; +} + /*[clinic input] _elementtree.XMLParser.feed @@ -4288,6 +4322,7 @@ static PyType_Spec treebuilder_spec = { static PyMethodDef xmlparser_methods[] = { _ELEMENTTREE_XMLPARSER_FEED_METHODDEF _ELEMENTTREE_XMLPARSER_CLOSE_METHODDEF + _ELEMENTTREE_XMLPARSER_FLUSH_METHODDEF _ELEMENTTREE_XMLPARSER__PARSE_WHOLE_METHODDEF _ELEMENTTREE_XMLPARSER__SETEVENTS_METHODDEF {NULL, NULL} diff --git a/Modules/clinic/_elementtree.c.h b/Modules/clinic/_elementtree.c.h index 9622591a1aa855..10b2dd1c15f7fd 100644 --- a/Modules/clinic/_elementtree.c.h +++ b/Modules/clinic/_elementtree.c.h @@ -1169,6 +1169,23 @@ _elementtree_XMLParser_close(XMLParserObject *self, PyObject *Py_UNUSED(ignored) return _elementtree_XMLParser_close_impl(self); } +PyDoc_STRVAR(_elementtree_XMLParser_flush__doc__, +"flush($self, /)\n" +"--\n" +"\n"); + +#define _ELEMENTTREE_XMLPARSER_FLUSH_METHODDEF \ + {"flush", (PyCFunction)_elementtree_XMLParser_flush, METH_NOARGS, _elementtree_XMLParser_flush__doc__}, + +static PyObject * +_elementtree_XMLParser_flush_impl(XMLParserObject *self); + +static PyObject * +_elementtree_XMLParser_flush(XMLParserObject *self, PyObject *Py_UNUSED(ignored)) +{ + return _elementtree_XMLParser_flush_impl(self); +} + PyDoc_STRVAR(_elementtree_XMLParser_feed__doc__, "feed($self, data, /)\n" "--\n" @@ -1219,4 +1236,4 @@ _elementtree_XMLParser__setevents(XMLParserObject *self, PyObject *const *args, exit: return return_value; } -/*[clinic end generated code: output=218ec9e6a889f796 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=aed9f53eeb0404e0 input=a9049054013a1b77]*/ diff --git a/Modules/clinic/pyexpat.c.h b/Modules/clinic/pyexpat.c.h index a5b93e68598204..343cb91b975038 100644 --- a/Modules/clinic/pyexpat.c.h +++ b/Modules/clinic/pyexpat.c.h @@ -8,6 +8,53 @@ preserve #endif #include "pycore_modsupport.h" // _PyArg_UnpackKeywords() +PyDoc_STRVAR(pyexpat_xmlparser_SetReparseDeferralEnabled__doc__, +"SetReparseDeferralEnabled($self, enabled, /)\n" +"--\n" +"\n" +"Enable/Disable reparse deferral; enabled by default with Expat >=2.6.0."); + +#define PYEXPAT_XMLPARSER_SETREPARSEDEFERRALENABLED_METHODDEF \ + {"SetReparseDeferralEnabled", (PyCFunction)pyexpat_xmlparser_SetReparseDeferralEnabled, METH_O, pyexpat_xmlparser_SetReparseDeferralEnabled__doc__}, + +static PyObject * +pyexpat_xmlparser_SetReparseDeferralEnabled_impl(xmlparseobject *self, + int enabled); + +static PyObject * +pyexpat_xmlparser_SetReparseDeferralEnabled(xmlparseobject *self, PyObject *arg) +{ + PyObject *return_value = NULL; + int enabled; + + enabled = PyObject_IsTrue(arg); + if (enabled < 0) { + goto exit; + } + return_value = pyexpat_xmlparser_SetReparseDeferralEnabled_impl(self, enabled); + +exit: + return return_value; +} + +PyDoc_STRVAR(pyexpat_xmlparser_GetReparseDeferralEnabled__doc__, +"GetReparseDeferralEnabled($self, /)\n" +"--\n" +"\n" +"Retrieve reparse deferral enabled status; always returns false with Expat <2.6.0."); + +#define PYEXPAT_XMLPARSER_GETREPARSEDEFERRALENABLED_METHODDEF \ + {"GetReparseDeferralEnabled", (PyCFunction)pyexpat_xmlparser_GetReparseDeferralEnabled, METH_NOARGS, pyexpat_xmlparser_GetReparseDeferralEnabled__doc__}, + +static PyObject * +pyexpat_xmlparser_GetReparseDeferralEnabled_impl(xmlparseobject *self); + +static PyObject * +pyexpat_xmlparser_GetReparseDeferralEnabled(xmlparseobject *self, PyObject *Py_UNUSED(ignored)) +{ + return pyexpat_xmlparser_GetReparseDeferralEnabled_impl(self); +} + PyDoc_STRVAR(pyexpat_xmlparser_Parse__doc__, "Parse($self, data, isfinal=False, /)\n" "--\n" @@ -498,4 +545,4 @@ pyexpat_ErrorString(PyObject *module, PyObject *arg) #ifndef PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF #define PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF #endif /* !defined(PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF) */ -/*[clinic end generated code: output=48c4296e43777df4 input=a9049054013a1b77]*/ +/*[clinic end generated code: output=892e48e41f9b6e4b input=a9049054013a1b77]*/ diff --git a/Modules/expat/pyexpatns.h b/Modules/expat/pyexpatns.h index d45d9b6c457159..8ee03ef0792815 100644 --- a/Modules/expat/pyexpatns.h +++ b/Modules/expat/pyexpatns.h @@ -108,6 +108,7 @@ #define XML_SetNotStandaloneHandler PyExpat_XML_SetNotStandaloneHandler #define XML_SetParamEntityParsing PyExpat_XML_SetParamEntityParsing #define XML_SetProcessingInstructionHandler PyExpat_XML_SetProcessingInstructionHandler +#define XML_SetReparseDeferralEnabled PyExpat_XML_SetReparseDeferralEnabled #define XML_SetReturnNSTriplet PyExpat_XML_SetReturnNSTriplet #define XML_SetSkippedEntityHandler PyExpat_XML_SetSkippedEntityHandler #define XML_SetStartCdataSectionHandler PyExpat_XML_SetStartCdataSectionHandler diff --git a/Modules/pyexpat.c b/Modules/pyexpat.c index 62cd262a7885e9..f04f96bc2f7601 100644 --- a/Modules/pyexpat.c +++ b/Modules/pyexpat.c @@ -7,6 +7,7 @@ #include "pycore_pyhash.h" // _Py_HashSecret #include "pycore_traceback.h" // _PyTraceback_Add() +#include <stdbool.h> #include <stddef.h> // offsetof() #include "expat.h" #include "pyexpat.h" @@ -81,6 +82,12 @@ typedef struct { /* NULL if not enabled */ int buffer_size; /* Size of buffer, in XML_Char units */ int buffer_used; /* Buffer units in use */ + bool reparse_deferral_enabled; /* Whether to defer reparsing of + unfinished XML tokens; a de-facto cache of + what Expat has the authority on, for lack + of a getter API function + "XML_GetReparseDeferralEnabled" in Expat + 2.6.0 */ PyObject *intern; /* Dictionary to intern strings */ PyObject **handlers; } xmlparseobject; @@ -703,6 +710,40 @@ get_parse_result(pyexpat_state *state, xmlparseobject *self, int rv) #define MAX_CHUNK_SIZE (1 << 20) +/*[clinic input] +pyexpat.xmlparser.SetReparseDeferralEnabled + + enabled: bool + / + +Enable/Disable reparse deferral; enabled by default with Expat >=2.6.0. +[clinic start generated code]*/ + +static PyObject * +pyexpat_xmlparser_SetReparseDeferralEnabled_impl(xmlparseobject *self, + int enabled) +/*[clinic end generated code: output=5ec539e3b63c8c49 input=021eb9e0bafc32c5]*/ +{ +#if XML_COMBINED_VERSION >= 20600 + XML_SetReparseDeferralEnabled(self->itself, enabled ? XML_TRUE : XML_FALSE); + self->reparse_deferral_enabled = (bool)enabled; +#endif + Py_RETURN_NONE; +} + +/*[clinic input] +pyexpat.xmlparser.GetReparseDeferralEnabled + +Retrieve reparse deferral enabled status; always returns false with Expat <2.6.0. +[clinic start generated code]*/ + +static PyObject * +pyexpat_xmlparser_GetReparseDeferralEnabled_impl(xmlparseobject *self) +/*[clinic end generated code: output=4e91312e88a595a8 input=54b5f11d32b20f3e]*/ +{ + return PyBool_FromLong(self->reparse_deferral_enabled); +} + /*[clinic input] pyexpat.xmlparser.Parse @@ -1063,6 +1104,8 @@ static struct PyMethodDef xmlparse_methods[] = { #if XML_COMBINED_VERSION >= 19505 PYEXPAT_XMLPARSER_USEFOREIGNDTD_METHODDEF #endif + PYEXPAT_XMLPARSER_SETREPARSEDEFERRALENABLED_METHODDEF + PYEXPAT_XMLPARSER_GETREPARSEDEFERRALENABLED_METHODDEF {NULL, NULL} /* sentinel */ }; @@ -1158,6 +1201,11 @@ newxmlparseobject(pyexpat_state *state, const char *encoding, self->ns_prefixes = 0; self->handlers = NULL; self->intern = Py_XNewRef(intern); +#if XML_COMBINED_VERSION >= 20600 + self->reparse_deferral_enabled = true; +#else + self->reparse_deferral_enabled = false; +#endif /* namespace_separator is either NULL or contains one char + \0 */ self->itself = XML_ParserCreate_MM(encoding, &ExpatMemoryHandler, @@ -2019,6 +2067,11 @@ pyexpat_exec(PyObject *mod) #else capi->SetHashSalt = NULL; #endif +#if XML_COMBINED_VERSION >= 20600 + capi->SetReparseDeferralEnabled = XML_SetReparseDeferralEnabled; +#else + capi->SetReparseDeferralEnabled = NULL; +#endif /* export using capsule */ PyObject *capi_object = PyCapsule_New(capi, PyExpat_CAPSULE_NAME,

1 0

[3.11] Fix rendering of null character in ast.rst (GH-116080) (#116161)
by hauntsaninja Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/da34d55f7f62cfbda71758e6c6febf047e… commit: da34d55f7f62cfbda71758e6c6febf047ecd14e7 branch: 3.11 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: hauntsaninja <12621235+hauntsaninja(a)users.noreply.github.com> date: 2024-02-29T22:42:31Z summary: [3.11] Fix rendering of null character in ast.rst (GH-116080) (#116161) Fix rendering of null character in ast.rst (GH-116080) (cherry picked from commit c04a981ff414b101736688773dbe24f824ca32ce) Co-authored-by: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> files: M Doc/library/ast.rst diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 837d70be0a3200..35242878c129de 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -2045,7 +2045,7 @@ and classes for traversing abstract syntax trees: ``await`` as variable names. The lowest supported version is ``(3, 4)``; the highest is ``sys.version_info[0:2]``. - If source contains a null character ('\0'), :exc:`ValueError` is raised. + If source contains a null character (``\0``), :exc:`ValueError` is raised. .. warning:: Note that successfully parsing source code into an AST object doesn't

1 0

[3.12] Fix rendering of null character in ast.rst (GH-116080) (#116160)
by hauntsaninja Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/8d865f19bd9d21ee3cce515742db25f44d… commit: 8d865f19bd9d21ee3cce515742db25f44d167b0f branch: 3.12 author: Miss Islington (bot) <31488909+miss-islington(a)users.noreply.github.com> committer: hauntsaninja <12621235+hauntsaninja(a)users.noreply.github.com> date: 2024-02-29T22:42:25Z summary: [3.12] Fix rendering of null character in ast.rst (GH-116080) (#116160) Fix rendering of null character in ast.rst (GH-116080) (cherry picked from commit c04a981ff414b101736688773dbe24f824ca32ce) Co-authored-by: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> files: M Doc/library/ast.rst diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 770eabf0c4214c..793e9ee1c8e409 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -2185,7 +2185,7 @@ and classes for traversing abstract syntax trees: ``await`` as variable names. The lowest supported version is ``(3, 4)``; the highest is ``sys.version_info[0:2]``. - If source contains a null character ('\0'), :exc:`ValueError` is raised. + If source contains a null character (``\0``), :exc:`ValueError` is raised. .. warning:: Note that successfully parsing source code into an AST object doesn't

1 0

gh-115685: Type/values propagate for TO_BOOL in tier 2 (GH-115686)
by Fidget-Spinner Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/d01886c5c9e3a62921b304ba7e5145daaa… commit: d01886c5c9e3a62921b304ba7e5145daaa56d3cf branch: main author: Ken Jin <kenjin(a)python.org> committer: Fidget-Spinner <kenjin4096(a)gmail.com> date: 2024-03-01T06:13:38+08:00 summary: gh-115685: Type/values propagate for TO_BOOL in tier 2 (GH-115686) files: M Include/internal/pycore_uop_ids.h M Include/internal/pycore_uop_metadata.h M Python/bytecodes.c M Python/executor_cases.c.h M Python/optimizer_bytecodes.c M Python/optimizer_cases.c.h diff --git a/Include/internal/pycore_uop_ids.h b/Include/internal/pycore_uop_ids.h index 3c133d97b2f03e..8f71eab44d914d 100644 --- a/Include/internal/pycore_uop_ids.h +++ b/Include/internal/pycore_uop_ids.h @@ -216,42 +216,43 @@ extern "C" { #define _POP_JUMP_IF_FALSE 402 #define _POP_JUMP_IF_TRUE 403 #define _POP_TOP POP_TOP +#define _POP_TOP_LOAD_CONST_INLINE_BORROW 404 #define _PUSH_EXC_INFO PUSH_EXC_INFO -#define _PUSH_FRAME 404 +#define _PUSH_FRAME 405 #define _PUSH_NULL PUSH_NULL #define _RESUME_CHECK RESUME_CHECK -#define _SAVE_RETURN_OFFSET 405 -#define _SEND 406 +#define _SAVE_RETURN_OFFSET 406 +#define _SEND 407 #define _SEND_GEN SEND_GEN #define _SETUP_ANNOTATIONS SETUP_ANNOTATIONS #define _SET_ADD SET_ADD #define _SET_FUNCTION_ATTRIBUTE SET_FUNCTION_ATTRIBUTE #define _SET_UPDATE SET_UPDATE -#define _START_EXECUTOR 407 -#define _STORE_ATTR 408 -#define _STORE_ATTR_INSTANCE_VALUE 409 -#define _STORE_ATTR_SLOT 410 +#define _START_EXECUTOR 408 +#define _STORE_ATTR 409 +#define _STORE_ATTR_INSTANCE_VALUE 410 +#define _STORE_ATTR_SLOT 411 #define _STORE_ATTR_WITH_HINT STORE_ATTR_WITH_HINT #define _STORE_DEREF STORE_DEREF -#define _STORE_FAST 411 -#define _STORE_FAST_0 412 -#define _STORE_FAST_1 413 -#define _STORE_FAST_2 414 -#define _STORE_FAST_3 415 -#define _STORE_FAST_4 416 -#define _STORE_FAST_5 417 -#define _STORE_FAST_6 418 -#define _STORE_FAST_7 419 +#define _STORE_FAST 412 +#define _STORE_FAST_0 413 +#define _STORE_FAST_1 414 +#define _STORE_FAST_2 415 +#define _STORE_FAST_3 416 +#define _STORE_FAST_4 417 +#define _STORE_FAST_5 418 +#define _STORE_FAST_6 419 +#define _STORE_FAST_7 420 #define _STORE_FAST_LOAD_FAST STORE_FAST_LOAD_FAST #define _STORE_FAST_STORE_FAST STORE_FAST_STORE_FAST #define _STORE_GLOBAL STORE_GLOBAL #define _STORE_NAME STORE_NAME #define _STORE_SLICE STORE_SLICE -#define _STORE_SUBSCR 420 +#define _STORE_SUBSCR 421 #define _STORE_SUBSCR_DICT STORE_SUBSCR_DICT #define _STORE_SUBSCR_LIST_INT STORE_SUBSCR_LIST_INT #define _SWAP SWAP -#define _TO_BOOL 421 +#define _TO_BOOL 422 #define _TO_BOOL_ALWAYS_TRUE TO_BOOL_ALWAYS_TRUE #define _TO_BOOL_BOOL TO_BOOL_BOOL #define _TO_BOOL_INT TO_BOOL_INT @@ -262,12 +263,12 @@ extern "C" { #define _UNARY_NEGATIVE UNARY_NEGATIVE #define _UNARY_NOT UNARY_NOT #define _UNPACK_EX UNPACK_EX -#define _UNPACK_SEQUENCE 422 +#define _UNPACK_SEQUENCE 423 #define _UNPACK_SEQUENCE_LIST UNPACK_SEQUENCE_LIST #define _UNPACK_SEQUENCE_TUPLE UNPACK_SEQUENCE_TUPLE #define _UNPACK_SEQUENCE_TWO_TUPLE UNPACK_SEQUENCE_TWO_TUPLE #define _WITH_EXCEPT_START WITH_EXCEPT_START -#define MAX_UOP_ID 422 +#define MAX_UOP_ID 423 #ifdef __cplusplus } diff --git a/Include/internal/pycore_uop_metadata.h b/Include/internal/pycore_uop_metadata.h index 35340fe9ee1b63..7f921a6cd3f4c8 100644 --- a/Include/internal/pycore_uop_metadata.h +++ b/Include/internal/pycore_uop_metadata.h @@ -232,6 +232,7 @@ const uint16_t _PyUop_Flags[MAX_UOP_ID+1] = { [_CHECK_VALIDITY] = HAS_DEOPT_FLAG, [_LOAD_CONST_INLINE] = HAS_PURE_FLAG, [_LOAD_CONST_INLINE_BORROW] = HAS_PURE_FLAG, + [_POP_TOP_LOAD_CONST_INLINE_BORROW] = HAS_PURE_FLAG, [_LOAD_CONST_INLINE_WITH_NULL] = HAS_PURE_FLAG, [_LOAD_CONST_INLINE_BORROW_WITH_NULL] = HAS_PURE_FLAG, [_CHECK_GLOBALS] = HAS_DEOPT_FLAG, @@ -425,6 +426,7 @@ const char *const _PyOpcode_uop_name[MAX_UOP_ID+1] = { [_POP_EXCEPT] = "_POP_EXCEPT", [_POP_FRAME] = "_POP_FRAME", [_POP_TOP] = "_POP_TOP", + [_POP_TOP_LOAD_CONST_INLINE_BORROW] = "_POP_TOP_LOAD_CONST_INLINE_BORROW", [_PUSH_EXC_INFO] = "_PUSH_EXC_INFO", [_PUSH_FRAME] = "_PUSH_FRAME", [_PUSH_NULL] = "_PUSH_NULL", diff --git a/Python/bytecodes.c b/Python/bytecodes.c index 1d515098f6c7e9..095982d64f34d0 100644 --- a/Python/bytecodes.c +++ b/Python/bytecodes.c @@ -4043,6 +4043,11 @@ dummy_func( value = ptr; } + tier2 pure op (_POP_TOP_LOAD_CONST_INLINE_BORROW, (ptr/4, pop -- value)) { + Py_DECREF(pop); + value = ptr; + } + tier2 pure op(_LOAD_CONST_INLINE_WITH_NULL, (ptr/4 -- value, null)) { value = Py_NewRef(ptr); null = NULL; diff --git a/Python/executor_cases.c.h b/Python/executor_cases.c.h index 9ec1be9076a5a0..e84c33281ee640 100644 --- a/Python/executor_cases.c.h +++ b/Python/executor_cases.c.h @@ -3759,6 +3759,17 @@ break; } + case _POP_TOP_LOAD_CONST_INLINE_BORROW: { + PyObject *pop; + PyObject *value; + pop = stack_pointer[-1]; + PyObject *ptr = (PyObject *)CURRENT_OPERAND(); + Py_DECREF(pop); + value = ptr; + stack_pointer[-1] = value; + break; + } + case _LOAD_CONST_INLINE_WITH_NULL: { PyObject *value; PyObject *null; diff --git a/Python/optimizer_bytecodes.c b/Python/optimizer_bytecodes.c index aa19a50f09ff71..2b47381ec76db4 100644 --- a/Python/optimizer_bytecodes.c +++ b/Python/optimizer_bytecodes.c @@ -254,6 +254,67 @@ dummy_func(void) { } } + op(_TO_BOOL, (value -- res)) { + (void)value; + res = sym_new_type(ctx, &PyBool_Type); + OUT_OF_SPACE_IF_NULL(res); + } + + op(_TO_BOOL_BOOL, (value -- value)) { + if (sym_matches_type(value, &PyBool_Type)) { + REPLACE_OP(this_instr, _NOP, 0, 0); + } + else { + if(!sym_set_type(value, &PyBool_Type)) { + goto hit_bottom; + } + } + } + + op(_TO_BOOL_INT, (value -- res)) { + if (sym_is_const(value) && sym_matches_type(value, &PyLong_Type)) { + PyObject *load = _PyLong_IsZero((PyLongObject *)sym_get_const(value)) + ? Py_False : Py_True; + REPLACE_OP(this_instr, _POP_TOP_LOAD_CONST_INLINE_BORROW, 0, (uintptr_t)load); + OUT_OF_SPACE_IF_NULL(res = sym_new_const(ctx, load)); + } + else { + OUT_OF_SPACE_IF_NULL(res = sym_new_type(ctx, &PyBool_Type)); + } + if(!sym_set_type(value, &PyLong_Type)) { + goto hit_bottom; + } + } + + op(_TO_BOOL_LIST, (value -- res)) { + if(!sym_set_type(value, &PyList_Type)) { + goto hit_bottom; + } + OUT_OF_SPACE_IF_NULL(res = sym_new_type(ctx, &PyBool_Type)); + } + + op(_TO_BOOL_NONE, (value -- res)) { + if (sym_get_const(value) == Py_None) { + REPLACE_OP(this_instr, _POP_TOP_LOAD_CONST_INLINE_BORROW, 0, (uintptr_t)Py_False); + } + sym_set_const(value, Py_None); + OUT_OF_SPACE_IF_NULL(res = sym_new_const(ctx, Py_False)); + } + + op(_TO_BOOL_STR, (value -- res)) { + if (sym_is_const(value) && sym_matches_type(value, &PyUnicode_Type)) { + PyObject *load = sym_get_const(value) == &_Py_STR(empty) ? Py_False : Py_True; + REPLACE_OP(this_instr, _POP_TOP_LOAD_CONST_INLINE_BORROW, 0, (uintptr_t)load); + OUT_OF_SPACE_IF_NULL(res = sym_new_const(ctx, load)); + } + else { + OUT_OF_SPACE_IF_NULL(res = sym_new_type(ctx, &PyBool_Type)); + } + if(!sym_set_type(value, &PyUnicode_Type)) { + goto hit_bottom; + } + } + op(_LOAD_CONST, (-- value)) { // There should be no LOAD_CONST. It should be all // replaced by peephole_opt. diff --git a/Python/optimizer_cases.c.h b/Python/optimizer_cases.c.h index b34c57376df88e..9d7ebb80f62857 100644 --- a/Python/optimizer_cases.c.h +++ b/Python/optimizer_cases.c.h @@ -104,45 +104,90 @@ } case _TO_BOOL: { + _Py_UopsSymbol *value; _Py_UopsSymbol *res; - res = sym_new_unknown(ctx); - if (res == NULL) goto out_of_space; + value = stack_pointer[-1]; + (void)value; + res = sym_new_type(ctx, &PyBool_Type); + OUT_OF_SPACE_IF_NULL(res); stack_pointer[-1] = res; break; } case _TO_BOOL_BOOL: { + _Py_UopsSymbol *value; + value = stack_pointer[-1]; + if (sym_matches_type(value, &PyBool_Type)) { + REPLACE_OP(this_instr, _NOP, 0, 0); + } + else { + if(!sym_set_type(value, &PyBool_Type)) { + goto hit_bottom; + } + } break; } case _TO_BOOL_INT: { + _Py_UopsSymbol *value; _Py_UopsSymbol *res; - res = sym_new_unknown(ctx); - if (res == NULL) goto out_of_space; + value = stack_pointer[-1]; + if (sym_is_const(value) && sym_matches_type(value, &PyLong_Type)) { + PyObject *load = _PyLong_IsZero((PyLongObject *)sym_get_const(value)) + ? Py_False : Py_True; + REPLACE_OP(this_instr, _POP_TOP_LOAD_CONST_INLINE_BORROW, 0, (uintptr_t)load); + OUT_OF_SPACE_IF_NULL(res = sym_new_const(ctx, load)); + } + else { + OUT_OF_SPACE_IF_NULL(res = sym_new_type(ctx, &PyBool_Type)); + } + if(!sym_set_type(value, &PyLong_Type)) { + goto hit_bottom; + } stack_pointer[-1] = res; break; } case _TO_BOOL_LIST: { + _Py_UopsSymbol *value; _Py_UopsSymbol *res; - res = sym_new_unknown(ctx); - if (res == NULL) goto out_of_space; + value = stack_pointer[-1]; + if(!sym_set_type(value, &PyList_Type)) { + goto hit_bottom; + } + OUT_OF_SPACE_IF_NULL(res = sym_new_type(ctx, &PyBool_Type)); stack_pointer[-1] = res; break; } case _TO_BOOL_NONE: { + _Py_UopsSymbol *value; _Py_UopsSymbol *res; - res = sym_new_unknown(ctx); - if (res == NULL) goto out_of_space; + value = stack_pointer[-1]; + if (sym_get_const(value) == Py_None) { + REPLACE_OP(this_instr, _POP_TOP_LOAD_CONST_INLINE_BORROW, 0, (uintptr_t)Py_False); + } + sym_set_const(value, Py_None); + OUT_OF_SPACE_IF_NULL(res = sym_new_const(ctx, Py_False)); stack_pointer[-1] = res; break; } case _TO_BOOL_STR: { + _Py_UopsSymbol *value; _Py_UopsSymbol *res; - res = sym_new_unknown(ctx); - if (res == NULL) goto out_of_space; + value = stack_pointer[-1]; + if (sym_is_const(value) && sym_matches_type(value, &PyUnicode_Type)) { + PyObject *load = sym_get_const(value) == &_Py_STR(empty) ? Py_False : Py_True; + REPLACE_OP(this_instr, _POP_TOP_LOAD_CONST_INLINE_BORROW, 0, (uintptr_t)load); + OUT_OF_SPACE_IF_NULL(res = sym_new_const(ctx, load)); + } + else { + OUT_OF_SPACE_IF_NULL(res = sym_new_type(ctx, &PyBool_Type)); + } + if(!sym_set_type(value, &PyUnicode_Type)) { + goto hit_bottom; + } stack_pointer[-1] = res; break; } @@ -1789,6 +1834,14 @@ break; } + case _POP_TOP_LOAD_CONST_INLINE_BORROW: { + _Py_UopsSymbol *value; + value = sym_new_unknown(ctx); + if (value == NULL) goto out_of_space; + stack_pointer[-1] = value; + break; + } + case _LOAD_CONST_INLINE_WITH_NULL: { _Py_UopsSymbol *value; _Py_UopsSymbol *null;

1 0

Fix rendering of null character in ast.rst (#116080)
by hauntsaninja Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/c04a981ff414b101736688773dbe24f824… commit: c04a981ff414b101736688773dbe24f824ca32ce branch: main author: Shantanu <12621235+hauntsaninja(a)users.noreply.github.com> committer: hauntsaninja <12621235+hauntsaninja(a)users.noreply.github.com> date: 2024-02-29T14:09:06-08:00 summary: Fix rendering of null character in ast.rst (#116080) files: M Doc/library/ast.rst diff --git a/Doc/library/ast.rst b/Doc/library/ast.rst index 5ed6eb63c0fb62..d10f3f275c0eaf 100644 --- a/Doc/library/ast.rst +++ b/Doc/library/ast.rst @@ -2193,7 +2193,7 @@ and classes for traversing abstract syntax trees: is no guarantee that the parse (or success of the parse) is the same as when run on the Python version corresponding to ``feature_version``. - If source contains a null character ('\0'), :exc:`ValueError` is raised. + If source contains a null character (``\0``), :exc:`ValueError` is raised. .. warning:: Note that successfully parsing source code into an AST object doesn't

1 0

gh-71052: Fix several Android build issues (#115955)
by erlend-aasland Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/fa1d675309c6a08b0833cf25cffe476c61… commit: fa1d675309c6a08b0833cf25cffe476c6166aba3 branch: main author: Malcolm Smith <smith(a)chaquo.com> committer: erlend-aasland <erlend.aasland(a)protonmail.com> date: 2024-02-29T22:58:20+01:00 summary: gh-71052: Fix several Android build issues (#115955) This change is part of the work on PEP-738: Adding Android as a supported platform. * Remove the "1.0" suffix from libpython's filename on Android, which would prevent Gradle from packaging it into an app. * Simplify the build command in the Makefile so that libpython always gets given an SONAME with the `-Wl-h` argument, even if the SONAME is identical to the actual filename. * Disable a number of functions on Android which can be compiled and linked against, but always fail at runtime. As a result, the native _multiprocessing module is no longer built for Android. * gh-115390 (bee7bb331) added some pre-determined results to the configure script for things that can't be autodetected when cross-compiling; this change adds Android to these where appropriate. * Add a couple more pre-determined results for Android, and making them cover iOS as well. This means the --enable-ipv6 configure option will no longer be required on either platform. files: A Misc/NEWS.d/next/Build/2024-02-26-14-54-58.gh-issue-71052.XvFay1.rst M Lib/ctypes/__init__.py M Makefile.pre.in M Modules/_multiprocessing/multiprocessing.c M configure M configure.ac diff --git a/Lib/ctypes/__init__.py b/Lib/ctypes/__init__.py index 141142a57dcb3e..d54ee05b15f5bd 100644 --- a/Lib/ctypes/__init__.py +++ b/Lib/ctypes/__init__.py @@ -468,6 +468,8 @@ def LoadLibrary(self, name): if _os.name == "nt": pythonapi = PyDLL("python dll", None, _sys.dllhandle) +elif hasattr(_sys, "getandroidapilevel"): + pythonapi = PyDLL("libpython%d.%d.so" % _sys.version_info[:2]) elif _sys.platform == "cygwin": pythonapi = PyDLL("libpython%d.%d.dll" % _sys.version_info[:2]) else: diff --git a/Makefile.pre.in b/Makefile.pre.in index 7533a49b4392f0..ee65ecd918ce2a 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -859,11 +859,9 @@ $(LIBRARY): $(LIBRARY_OBJS) $(AR) $(ARFLAGS) $@ $(LIBRARY_OBJS) libpython$(LDVERSION).so: $(LIBRARY_OBJS) $(DTRACE_OBJS) - if test $(INSTSONAME) != $(LDLIBRARY); then \ - $(BLDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM); \ + $(BLDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM) + if test $(INSTSONAME) != $@; then \ $(LN) -f $(INSTSONAME) $@; \ - else \ - $(BLDSHARED) -o $@ $(LIBRARY_OBJS) $(MODLIBS) $(SHLIBS) $(LIBC) $(LIBM); \ fi libpython3.so: libpython$(LDVERSION).so diff --git a/Misc/NEWS.d/next/Build/2024-02-26-14-54-58.gh-issue-71052.XvFay1.rst b/Misc/NEWS.d/next/Build/2024-02-26-14-54-58.gh-issue-71052.XvFay1.rst new file mode 100644 index 00000000000000..bda91335814936 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2024-02-26-14-54-58.gh-issue-71052.XvFay1.rst @@ -0,0 +1 @@ +Fix several Android build issues diff --git a/Modules/_multiprocessing/multiprocessing.c b/Modules/_multiprocessing/multiprocessing.c index 2e6d8eb68c0243..1f6ab718a36984 100644 --- a/Modules/_multiprocessing/multiprocessing.c +++ b/Modules/_multiprocessing/multiprocessing.c @@ -181,7 +181,7 @@ static PyMethodDef module_methods[] = { _MULTIPROCESSING_RECV_METHODDEF _MULTIPROCESSING_SEND_METHODDEF #endif -#if !defined(POSIX_SEMAPHORES_NOT_ENABLED) && !defined(__ANDROID__) +#if !defined(POSIX_SEMAPHORES_NOT_ENABLED) _MULTIPROCESSING_SEM_UNLINK_METHODDEF #endif {NULL} diff --git a/configure b/configure index f431c5dd15ec4a..45f232164e7094 100755 --- a/configure +++ b/configure @@ -7423,7 +7423,13 @@ printf "%s\n" "#define Py_ENABLE_SHARED 1" >>confdefs.h LDLIBRARY='libpython$(LDVERSION).so' BLDLIBRARY='-L. -lpython$(LDVERSION)' RUNSHARED=LD_LIBRARY_PATH=`pwd`${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} - INSTSONAME="$LDLIBRARY".$SOVERSION + + # The Android Gradle plugin will only package libraries whose names end + # with ".so". + if test "$ac_sys_system" != "Linux-android"; then + INSTSONAME="$LDLIBRARY".$SOVERSION + fi + if test "$with_pydebug" != yes then PY3LIBRARY=libpython3.so @@ -13726,7 +13732,14 @@ then : else $as_nop if test "$cross_compiling" = yes then : + +# "yes" changes the hash function to FNV, which causes problems with Numba +# (https://github.com/numba/numba/blob/0.59.0/numba/cpython/hashing.py#L470). +if test "$ac_sys_system" = "Linux-android"; then + ac_cv_aligned_required=no +else ac_cv_aligned_required=yes +fi else $as_nop cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ @@ -17529,6 +17542,23 @@ else printf "%s\n" "$MACHDEP_OBJS" >&6; } fi +if test "$ac_sys_system" = "Linux-android"; then + # When these functions are used in an unprivileged process, they crash rather + # than returning an error. + privileged_funcs="chroot initgroups setegid seteuid setgid setregid setresgid + setresuid setreuid setuid" + + # These functions are unimplemented and always return an error. + unimplemented_funcs="sem_open sem_unlink" + + for name in $privileged_funcs $unimplemented_funcs; do + as_func_var=`printf "%s\n" "ac_cv_func_$name" | $as_tr_sh` + + eval "$as_func_var=no" + + done +fi + # checks for library functions ac_fn_c_check_func "$LINENO" "accept4" "ac_cv_func_accept4" if test "x$ac_cv_func_accept4" = xyes @@ -22012,10 +22042,11 @@ fi done -# On iOS, clock_settime can be linked (so it is found by -# configure), but it raises a runtime error if used because apps can't change -# the clock. Force the symbol off. -if test "$ac_sys_system" != "iOS" ; then +# On Android and iOS, clock_settime can be linked (so it is found by +# configure), but when used in an unprivileged process, it crashes rather than +# returning an error. Force the symbol off. +if test "$ac_sys_system" != "Linux-android" && test "$ac_sys_system" != "iOS" +then for ac_func in clock_settime do : @@ -22295,7 +22326,9 @@ else $as_nop if test "$cross_compiling" = yes then : -if test "${enable_ipv6+set}" = set; then +if test "$ac_sys_system" = "Linux-android" || test "$ac_sys_system" = "iOS"; then + ac_cv_buggy_getaddrinfo="no" +elif test "${enable_ipv6+set}" = set; then ac_cv_buggy_getaddrinfo="no -- configured with --(en|dis)able-ipv6" else ac_cv_buggy_getaddrinfo=yes @@ -26968,7 +27001,7 @@ CPPFLAGS=$ac_save_cppflags { printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for device files" >&5 printf "%s\n" "$as_me: checking for device files" >&6;} -if test "$ac_sys_system" = "iOS" ; then +if test "$ac_sys_system" = "Linux-android" || test "$ac_sys_system" = "iOS"; then ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no else diff --git a/configure.ac b/configure.ac index a5abb06a829413..b7671facc65a8a 100644 --- a/configure.ac +++ b/configure.ac @@ -1443,7 +1443,13 @@ if test $enable_shared = "yes"; then LDLIBRARY='libpython$(LDVERSION).so' BLDLIBRARY='-L. -lpython$(LDVERSION)' RUNSHARED=LD_LIBRARY_PATH=`pwd`${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} - INSTSONAME="$LDLIBRARY".$SOVERSION + + # The Android Gradle plugin will only package libraries whose names end + # with ".so". + if test "$ac_sys_system" != "Linux-android"; then + INSTSONAME="$LDLIBRARY".$SOVERSION + fi + if test "$with_pydebug" != yes then PY3LIBRARY=libpython3.so @@ -3675,7 +3681,14 @@ int main(void) }]])], [ac_cv_aligned_required=no], [ac_cv_aligned_required=yes], -[ac_cv_aligned_required=yes]) +[ +# "yes" changes the hash function to FNV, which causes problems with Numba +# (https://github.com/numba/numba/blob/0.59.0/numba/cpython/hashing.py#L470). +if test "$ac_sys_system" = "Linux-android"; then + ac_cv_aligned_required=no +else + ac_cv_aligned_required=yes +fi]) ]) if test "$ac_cv_aligned_required" = yes ; then AC_DEFINE([HAVE_ALIGNED_REQUIRED], [1], @@ -4872,6 +4885,22 @@ else AC_MSG_RESULT([$MACHDEP_OBJS]) fi +if test "$ac_sys_system" = "Linux-android"; then + # When these functions are used in an unprivileged process, they crash rather + # than returning an error. + privileged_funcs="chroot initgroups setegid seteuid setgid setregid setresgid + setresuid setreuid setuid" + + # These functions are unimplemented and always return an error. + unimplemented_funcs="sem_open sem_unlink" + + for name in $privileged_funcs $unimplemented_funcs; do + AS_VAR_PUSHDEF([func_var], [ac_cv_func_$name]) + AS_VAR_SET([func_var], [no]) + AS_VAR_POPDEF([func_var]) + done +fi + # checks for library functions AC_CHECK_FUNCS([ \ accept4 alarm bind_textdomain_codeset chmod chown clock closefrom close_range confstr \ @@ -5216,10 +5245,11 @@ AC_CHECK_FUNCS([clock_getres], [], [ ]) ]) -# On iOS, clock_settime can be linked (so it is found by -# configure), but it raises a runtime error if used because apps can't change -# the clock. Force the symbol off. -if test "$ac_sys_system" != "iOS" ; then +# On Android and iOS, clock_settime can be linked (so it is found by +# configure), but when used in an unprivileged process, it crashes rather than +# returning an error. Force the symbol off. +if test "$ac_sys_system" != "Linux-android" && test "$ac_sys_system" != "iOS" +then AC_CHECK_FUNCS([clock_settime], [], [ AC_CHECK_LIB([rt], [clock_settime], [ AC_DEFINE([HAVE_CLOCK_SETTIME], [1]) @@ -5371,7 +5401,9 @@ int main(void) [ac_cv_buggy_getaddrinfo=no], [ac_cv_buggy_getaddrinfo=yes], [ -if test "${enable_ipv6+set}" = set; then +if test "$ac_sys_system" = "Linux-android" || test "$ac_sys_system" = "iOS"; then + ac_cv_buggy_getaddrinfo="no" +elif test "${enable_ipv6+set}" = set; then ac_cv_buggy_getaddrinfo="no -- configured with --(en|dis)able-ipv6" else ac_cv_buggy_getaddrinfo=yes @@ -6589,9 +6621,9 @@ CPPFLAGS=$ac_save_cppflags AC_MSG_NOTICE([checking for device files]) dnl NOTE: Inform user how to proceed with files when cross compiling. -dnl iOS cross-compile builds are predictable; they won't ever +dnl Some cross-compile builds are predictable; they won't ever dnl have /dev/ptmx or /dev/ptc, so we can set them explicitly. -if test "$ac_sys_system" = "iOS" ; then +if test "$ac_sys_system" = "Linux-android" || test "$ac_sys_system" = "iOS"; then ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no else

1 0

gh-87115: Set `__main__.__spec__` to `None` in pdb (#116141)
by iritkatriel Feb. 29, 2024

Feb. 29, 2024

https://github.com/python/cpython/commit/ccfc042bbf31e53c44b8aae444afd8365b… commit: ccfc042bbf31e53c44b8aae444afd8365b798422 branch: main author: Tian Gao <gaogaotiantian(a)hotmail.com> committer: iritkatriel <1055913+iritkatriel(a)users.noreply.github.com> date: 2024-02-29T21:39:50Z summary: gh-87115: Set `__main__.__spec__` to `None` in pdb (#116141) files: A Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst M Lib/pdb.py M Lib/test/test_pdb.py diff --git a/Lib/pdb.py b/Lib/pdb.py index 0754e8b628cf57..519c1ccd5640a1 100755 --- a/Lib/pdb.py +++ b/Lib/pdb.py @@ -188,6 +188,7 @@ def namespace(self): __name__='__main__', __file__=self, __builtins__=__builtins__, + __spec__=None, ) @property diff --git a/Lib/test/test_pdb.py b/Lib/test/test_pdb.py index 2b0795cdad707e..3dd275caf43200 100644 --- a/Lib/test/test_pdb.py +++ b/Lib/test/test_pdb.py @@ -2695,6 +2695,18 @@ def bœr(): ('bœr', 2), ) + def test_spec(self): + # Test that __main__.__spec__ is set to None when running a script + script = """ + import __main__ + print(__main__.__spec__) + """ + + commands = "continue" + + stdout, _ = self.run_pdb_script(script, commands) + self.assertIn('None', stdout) + def test_find_function_first_executable_line(self): code = textwrap.dedent("""\ def foo(): pass diff --git a/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst b/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst new file mode 100644 index 00000000000000..844340583cd456 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-02-29-20-06-06.gh-issue-87115.FVMiOR.rst @@ -0,0 +1 @@ +Set ``__main__.__spec__`` to ``None`` when running a script with :mod:`pdb`

1 0