9.0.0: Release 9.0 (#1483)

Added support for Python 3.13. Dropped support for Python 3.8 and 3.9.

Note: There are incompatible changes. Use with Mathics-scanner 2.0.0 or greater.

You may notice a speedup in performance, especially graphics performance, in this version. There is a speedup due to removing conversions from Mathics3 to Python and vice versa for literal data, which happens a lot in plotting graphics. Also, Python 3.13 is a bit faster than previous versions. Previously, rendering via asymptote was slow. This is no longer the situation.

Preliminary work to track locations has started. This is useful in debugging and error reporting, and is controlled via Boolean System variable $TrackLocations.

Boxing operators have been added. The full range of escape sequences is supported. A limited form of boxing escape \* that handles a single Boxing function has been added.

A basic interrupt handler was added that loosely follows wolframscript's interrupt handler. Interrupt commands "abort", "exit", "continue", "debugger", "show", and "inspect" are available; "trace" will be added later.

main.py has been moved to __main__.py following Python conventions for main routines. This makes python -m mathics work. GNU Readline history is enabled for mathics when it is available. It shares history files with mathicsscript.

The priority for rule selection when there are several matching a function call has been revised and more closely follows WMA behavior.

Assignment statements, e.g. SetDelayed, UpSetDelayed, or DownSetDelayed have been revised to isolate left-hand-side references from conditions and element attributes. As a result, more of the code in WMA and Mathics3 packages work.

$IterationLimit detects runaway rule expansion better.

Parameter count checking expanded to more Builtin functions.

New Builtins

• $SessionID
• $TrackLocations (not WMA)
• BinaryReadList[] (needed to support importing gzip files)
• Hypergeometric2F1

By Aravindh Krishnamoorthy (needed for better Rubi support):

• Hypergeometric1F1[]
• HypergeometricPFQ[]
• MeijerG[]
• HypergeometricU[]

Documentation

Go over docs for Beta[], Gamma[], Product[], and infix operators with no meaning.
Expand Transpose[] documentation.

Enhancements

• Set-related code reworked for better WMA conformance. There is better WMA conformance in rule selection when several rules match.
• mathics CLI options are more like wolframscript
• The debugging interface has been improved. TraceEvaluation[] and TraceDebug[] filter and colorize output for Mathics3 constructs much better.
• Single-dash long options like -help, -file are now accepted. Short option -f is associated with -file rather than --fullform; -F is is now used for
  FullForm. Option --read with alias -r is now -code and short option -c.
• Boolean Options ShowRewrites and ShowEvaluation were added to TraceEvalation[]. These filtering for either rewrite rules or evaluation expressions. Presumably, you don't want to filter both.
• We check argument counts on more Builtin Functions and give error messages (tags argb, argx, argr, argrx) for invalid parameter combinations.
• $TraceBuiltins output uses standard Mathics3 I/O mechanisms rather than Python's builtin print. Therefore it will be seen in more front-ends like Django or pyoxide.

Bugs Fixed

• #1057 ListPlot[] error handling (and NestList[]) needs going over
• #1213 Condition[] expressions as second element in RuleDelayed behaviour not compatible with WMA
• #1187 Add Hypergeometric2F1 Builtin Function
• #1198 Blanks in Set operations are not properly handled in tag positions.
• #1245 Add "lpn" error message checking in _ListPlot
• #1383 Support for hypergeometric functions
• #1384 Option management tweaks
• #1388 In WMA, Pochhammer[0,-2] returns 1/2
• #1395 Match WMA for Gamma[1+x] and Product[...]
• #1405 structure_cache in mathics.core.expression.structure is None but we try to set it in _is_neutral_symbol()
• #1412 Transpose[] does not work on three-dimensional array
• #1425 `Erroneous Protected message in SetDelayed
• #1432 URL links with $ in them are getting messed up
• #1461 "noopen" errors sometimes return $Failed
• #1465 Crash in running Trace[Sin[Log[2.5, 7]]]
• #1473 Doctest for Quantity, KnownUnitQ, and others fail when the documentation is generated
• #1474 Document typo: "is a valid Association object" should be "is a valid Quantity object"
• #1476 $IterationLimit is not limiting evalation expansion

WMA Compatibility

• Hypergeometric functions have been revised to conform better to WMA behavior by expanding hypergeometric results.
• $IterationLimit now defaults to 4096.
• mathics command-line conform better to wolframscript options.
• Rule selection of functions when multiple rules apply conforms to WMA more closely.
• LHS reference selection conforms to WMA more closely.

Incompatible changes

Scanner API has changed. Options on mathics CLI have changed. See above for the changes.
Location of mathics in mathics.__main__, the more usual location, rather than mathics.main.

• Mathics scanner exceptions of class TranslateError are incompatible
  with previous versions, and now store error parameters, "name", "tag", and
  "args".
• The method get_sort_key() was replaced by two different properties:
  element_order, for canonical ordering of expressions, and
  pattern_precedence, used for ordering rules according to their precedence
  in the evaluation loop.
• In both cases, the part of the sort key related to properties of the
  expressions and patterns are now stored as a magic number instead of
  a tuple.

8.0.1

Some work was made on the Mathics3 Kernel to work in Python 3.13.
The maximum version of numpy was increased to < 2.3 to allow marimo to work.

Bugs

Correct for mismatch between ListExpression and tuple in DispatchAtom.
This is needed for PacletManager code to work better.

Compatibility

• When the result of an evaluation is Symbol`Null, Mathics CLI now does not show an Out[...]= line, following the behavior of
  the WMA CLI.
• Asymptote rendering of platonic solids added.

Internals

Document tagging code handles TeX math mode more completely. Image tags in PDF properly.

Documentation

• Documentation has been gone over so that expressions are tagged in TeX. As a result the user guide and reference manual render much nicer in the PDF and Django.
• More links have been added. References to The Digital Library of Mathematical Functions https://dlmf.nist.gov/ have been added where appropriate.
• Add mention of MathicsLive
• Platonic solid rendered properly in PDF

8.0.0

This release is to get out some of the major changes that have gone on already in advance of redoing Boxing and Formatting.

Code now supports the emscripten platform, so this code can be installed in pyodide using micropip.install.

Operators are now controlled from a new operators YAML table from the mathics-scanner repository. A pass was made over the Mathics parser to handle box operators more properly. More work is needed here.

We started adding more debugging capabilities:

• Breakpoint[]
• Stack[], and
• Trace[]

And in the Mathics3-Trepan repository:

• DebugActivate[]
• Debugger[], and
• TraceActivate[]

Option --post-mortem was added which goes into the trepan3k debugger on an unrecoverable error. This option is available on other front ends.

This debugging code is very much alpha quality, but it greatly improves the ability to debug problems in loading existing packages
written from Mathematica. So packages BoolEval and CleanSlate were added to the repository.

Also as a result of the improved ability to debug Mathics3, we now provide a version of Rubi 4.17 using git submodules . To use this you will need a patched version of stopit. Aravindh Krishnamoorthy led the initial port of Rubi.

David A. Roberts worked on ensuring Mathics3 runs on pyodide and contributed a number of new Built-in Functions that are found in The On-Line Encyclopedia of Integer Sequences (OEIS).

New Builtins

• Between
• Breakpoint - (not WMA; forces a Python breakpoint()
• CheckAbort
• FileNameDrop
• FormatValues
• ListStepPlot
• MapApply
• PythonCProfileEvaluation (not WMA; interface to Python cProfile)
• RealValuedNumberQ
• SequenceForm
• SetEnvironment
• Stack
• SyntaxQ
• Trace
• UnitStep

By @davidar <https://github.com/davidar>_:

• BellB
• DivisorSigma
• DivisorSum
• EulerE
• HypergeometricU
• IntegerPart
• IntegerPartitions
• JacobiSymbol
• KroneckerSymbol
• LambertW
• LinearRecurrence
• LucasL
• MersennePrimeExponent
• MoebiusMu
• NumberDigit
• PolygonalNumber
• PolyLog
• PowersRepresentations
• ReverseSort
• RootSum
• SeriesCoefficient
• SquaresR
• Subfactorial

Documentation

• Unicode operators appear in Django documentation. In the PDF, AMSLaTeX is used.
• Summaries of builtin functions have been improved and regularized

mathics command line

Option --post-mortem was added which goes into the trepan3k debugger <https https://pypi.org/project/trepan3k/>_ on an
unrecoverable error. This option is available on other front-ends..

WMA Compatibility

• GetEnvironment expanded to handle [] and {var1, var2,...} forms
• The system packages directory has been renamed Packages to conformance with WMA.
• $Path now includes a Packages directory under $HOME.
• All of the 100 or so Unicode operators without a pre-defined meaning are now supported

Internals

• More of the on-OO evaluation code that forms what might be an
  instruction evaluator has been moved out of the module
  mathics.builtins put in mathics.eval. This includes code for plotting and making boxes.
• nested TimeConstraint[] works via external Python module stopit.
• Pause[] is more interruptable
• More code has been linted, more type errors removed, and docstrings added/improved

Performance

• Blank* patterns without arguments are now singletons.

API incompatibility

• Matcher now requires an additional evaluation parameter
• Romberg removed as an NIntegrate[] method. It is deprecated in SciPy and is to be removed by SciPy 1.15.
• The signature of the Definition.__init__ now receives a single dict parameter instead of the several *values parameters.
• Rule positions in Definition.{get|set}_values now includes the word values. For example pos="up" now is pos="upvalues".
• Definitions.get_ownvalue now returns a BaseElement instead of a BaseRule object.
• Patterns in eval_ and format_ methods of builtin classes
  parses patterns in docstrings of the form
  Symbol: Expr as Pattern[Symbol, Expr].
  To specify the associated format in format_ methods the
  docstring, the list of format must be wrapped in parenthesis, like
  (InputForm,): Definitions[...] instead of just InputForm: Definitions[...].
• Character and Operator information that has been gone over in the Mathics Scanner project. The information in JSON tables, the keys, and values have thus change. Here, we read this information in and use that instead of previously hard-coded values.

Bugs

• Fix infinite recursion when formatting Sequence[...]
• Parsing \( ... \) improved
• Fixed #1105, #1106, #1107, #1172 #1173, #1195, #1205, #1221, #1223, and #1228 among others

Mathics3 Packages

• Added BoolEval
• Added CleanSlate
• Combinatorica moved to a separate repository and v.9 was renamed to 0.9.1.
  More code v0.9.1 works. v2.0 was renamed v2.0.1 and some code now works.
• Rubi version 4.17 (work in progress; algebraic integrations work)

Mathics3 Modules

• Added preliminary Mathics3 debugger.

Python Package Updates

• Python 3.12 is now supported
• SymPy 1.13 is now supported

7.0.0

Some work was done here in support of planned future improvements like lazy loading of builtin functions. A bit of effort was also spent to modernize Python code and style, add more type annotations, remove spelling errors, and use newer versions of important software like SymPy and Python itself.

New Builtins

• $MaxLengthIntStringConversion
• Elements
• ComplexExpand (thanks to vitrun)
• ConjugateTranspose
• LeviCivitaTensor
• RealAbs and RealSign
• RealValuedNumberQ

Documentation

Many formatting issues with the PDF file have been addressed. In particular, the spacing of section numbers in chapter and section table of contents has been increased. The margin space around builtin definitions has a also been increased. Numerous spelling corrections to the document have been applied.

The code to run doctests and produce LaTeX documentation has been revised and refactored to allow incremental builtin update, and to DRY the code.

Section Head-Related Operations is a new section off of "Expression Structure". The title of the PDF has changed from Mathics to Mathics3 and the introduction has been updated and revised.

Compatibility

• *Plot does not show messages during the evaluation.
• Range[] now handles a negative di PR #951
• Improved support for DirectedInfinity and Indeterminate.
• Graphics and Graphics3D including wrong primitives and
  directives are shown with a pink background. In the Mathics-Django
  interface, a tooltip error message is also shown.
• Improving support for $CharacterEncoding. Now it is possible to
  change it from inside the session.

Internals ---

• eval_abs and eval_sign extracted from Abs and Sign and added to mathics.eval.arithmetic.
• Maximum number of digits allowed in a string set to 7000 and can be adjusted using environment variable MATHICS_MAX_STR_DIGITS on Python versions that don't adjust automatically (like pyston).
• Real number comparisons implemented is based now in the internal implementation of RealSign.
• For Python 3.11, the variable $MaxLengthIntStringConversion controls the maximum size of the literal conversion between large integers and Strings.
• Older style non-appearing and non-pedagogical doctests have been converted to pytest
• Built-in code is directed explicitly rather than implicitly. This facilitates the ability to lazy load builtins or "autoload" them via GNU Emacs autoload.
• add mpmath lru cache
• Some works was done to make it possible so that in the future we can speed up initial loading and reduce the initial memory footprint

Bugs

• Definitions is compatible with pickle.
• Improved support for Quantity expressions, including conversions, formatting and arithmetic operations.
• Background option for Graphics and Graphics3D is operative again.
• Numeric comparisons against expressions involving Strings; Issue #797)
• Switch[] involving Infinity. Issue #956
• Outer[] on SparseArray. Issue #939
• ArrayQ[] detects SparseArray PR #947
• BoxExpressionError exceptions handled Issue. PR #970
• Derivative evaluation of True, False and List[] corrected. PR #971, #973
• Combinatorica package fixes. PR #974
• Exit[] not working. PR #998
• BaseForm is now listed as in $OutputForms

API

We now require an explicit call to a new functionimport_and_load_builtins(). Previously loading was implicit and
indeterminate as to when this occurred as it was based on import order.

We need this so that we can support in the future lazy loading of builtin modules.

Package updates

1. Python 3.11 is now supported
2. Sympy 1.12 is now supported

6.0.4

Fixed botched 6.0.3 release:

Correct type annotation in mathics.session.MathicsSession

See #934

6.0.3

Correct type annotation in mathics.session.MathicsSession

See #934

OpenSUSE Tumbleweed

Change testing to facilitate openSUSE Tumbleweed distribution which uses Sympy 1.12. See Issue #881.

6.0.1

Release to get Pillow 9.2 dependency added for Python 3.7+

Note that future releases in the 6.1 or 7.0 range will drop support for Python 3.6.

Some Pattern-matching code gone over to add type annotations and to start documenting its behavior and characteristics. Function
attributes are now examined and stored at the time of Pattern-object creation rather than at evaluation time. This better matches WMA behavior which pulls out attribute this even earlier than this. These changes speed up doctest running time by about 7% under Pyston.

Combinatorica version upgraded from 0.9 (circa 1992) to 0.91 (circa 1995) which closer matches the published book.

Random Builtin function documentation gone over to conform to current documentation style.

Too late for Valentines Day

A fair bit of code refactoring has gone on so that we might be able to scale the code, get it to be more performant, and more in line with other interpreters. There is greater use of Symbols as opposed to strings.

The builtin Functions have been organized into grouping akin to what is found in WMA. This is not just for documentation purposes, but it better modularizes the code and keep the modules smaller while suggesting where functions below as we scale.

Image Routines have been gone over and fixed. Basically we use Pillow imaging routines and as opposed to home-grown image code.

A number of Built-in functions that were implemented were not accessible for various reasons.

Mathics3 Modules are better integrated into the documentation. Existing Mathics3 modules pymathics.graph and pymathics.natlang have had a major overhaul, although more is needed. And will continue after the 6.0.0 release

We have gradually been rolling in more Python type annotations and current Python practices such as using isort, black and flake8.

Evaluation methods of built-in functions start eval_ not apply_.

API

1. New function mathics.system_info.python_implementation() shows the Python Implementation, e.g. CPython, PyPy, Pyston that is running Python. This is included in the information mathics.system_info.mathics_system__system_info() returns and is used in $PythonImplementation
2. A list of optional software can be found in mathics.optional_software. Versions of that software are included in mathics.version_info.

Package update

• SymPy 1.11.1 accepted
• Numpy 1.24.0 accepted

New Builtins

• $BoxForms
• $OutputForms
• $PrintForms
• $PythonImplementation
• Accuracy
• ClebschGordan
• Curl (2-D and 3-D vector forms only)
• DiscretePlot
• Kurtosis
• ListLogPlot
• LogPlot
• $MaxMachineNumber
• $MinMachineNumber
• NumberLinePlot
• PauliMatrix
• Remove
• SetOptions
• SixJSymbol
• Skewness
• ThreeJSymbol

Documentation

1. All Builtins have links to WMA pages.
2. "Accuracy and Precision" section added to the Tutorial portion.
3. "Attribute Definitions" section reinstated.
4. "Expression Structure" split out as a guide section (was "Structure of Expressions").
5. "Exponential Functional" split out from "Trigonometry Functions"
6. "Functional Programming" section split out.
7. "Image Manipulation" has been split off from Graphics and Drawingand turned into a guide section.
8. Image examples now appear in the LaTeX and therfore the PDF doc
9. "Logic and Boolean Algebra" section reinstated.
10. "Forms of Input and Output" is its own guide section.
11. More URL links to Wiki pages added; more internal cross links added.
12. "Units and Quantities" section reinstated.
13. The Mathics3 Modules are now included in LaTeX and therefore the PDF doc.

Internals

• boxes_to_ methods are now optional for BoxElement subclasses. Most of the code is now moved to the mathics.format submodule, and implemented in a more scalable way.
• from_mpmath conversion supports a new parameter acc to set the accuracy of the number.
• mathics.builtin.inout was split in several modules (inout, messages, layout, makeboxes) in order to improve the documentation.
• mathics.eval was create to have code that might be put in an instruction interpreter. The opcodes-like functions start eval_, other functions are helper functions for those.
• Operator name to Unicode or ASCII comes from Mathics scanner character tables.
• Builtin instance methods that start eval are considered rule matching and function application; the use of the name applyis deprecated, when eval is intended.
• Modularize and improve the way in which Builtin classes are selected to have an associated Definition.
• _SetOperator.assign_elementary was renamed _SetOperator.assign. All the special cases are not handled by the _SetOperator.special_cases dict.
• isort run over all Python files. More type annotations and docstrings on functions added.
• caching on immutable atoms like, String, Integer, Real, etc. was improved; the __hash__() function was sped up. There is asmall speedup overall from this at the expense of increased memory.
• more type annotations added to functions, especially builtin functions
• Numerical constants used along the code was renamed using caps, according to the Python's convention.

Bugs

• 0 with a given precision (like in 0`3) is now parsed as 0, an integer number.
• Reading certain GIFs now work again
• Random[] works now.
• RandomSample with one list argument now returns a random ordering of the list items. Previously it would return just one item.
• Origin placement corrected on ListPlot and LinePlot.
• Fix long-standing bugs in Image handling
• Some scikit image routines line EdgeDetect were getting omitted due to overly stringent PyPI requirements
• Units and Quantities were sometimes failing. Also they were omitted from documentation.
• Better handling of Infinite quantities.
• Improved Precision and Accuracycompatibility with WMA. In particular, Precision[0.] and Accuracy[0.]
• Accuracy in numbers using the notation n.nnn``acc now is properly handled.
• numeric precision in mpmath was not reset after operations that changed these. This cause huges slowdowns after an operation that set the mpmath precison high. This was the source of several-minute slowdowns in testing.
• GIF87a (```MadTeaParty.gif`` or ExampleData) image loading fixed
• Replace non-free Leena image with a a freely distributable image. Issue #728

PyPI Package requirements

Mathics3 aims at a more richer set of functionality.

Therefore NumPy and Pillow (9.10 or later) are required Python packages where they had been optional before. In truth, probably running Mathics without one or both probably did not work well if it worked at all; we had not been testing setups that did not have NumPy.

Enhancements

• Vector restriction on Norm[] removed. "Frobinius" p-form allowed.
• Better handling of comparisons with finite precision numbers.
• Improved implementation for Precision.
• Infix operators, like -> render with their Unicode symbol when $CharacterEncoding is not "ASCII".
  *Grid compatibility with WMA was improved. Now it supports non-uniform list of lists and lists with general elements.
• Support for BigEndian Big TIFF

5.0.2

Rerelease to get requirements-cython.txt into tarball.