mathics
Things needed to make FeynCalc work
- System`$Notebook should be set to False -> OK
- System`$VersionNumber=10.
Symbols to implement
- System`FileNames (Implemented in PR #1173)
- System`URLSave (Implemented in PR #1173)
- System`CreateTemporary (Implemented in PR #1173)
- ToString[expr, format] (second argument)
- Use
Conditionas a pattern that can be lhs in assigment. - ???
NamedCharacters to implement
-
\[IndentingNewLine]
At the moment the issue with Read
https://github.com/mathics/Mathics/issues/1169#issuecomment-805786663
seems to be the most pressing one.
System`$Notebooks can also be False, since FeynCalc can work without the FrontEnd.
Regarding the $VersionNumber, I'll probably just tweak the loader to recognize Mathics
and disable the version check for that case.
At the moment the issue with
Readseems to be the most pressing one.
System`$Notebookscan also beFalse, since FeynCalc can work without the FrontEnd. Regarding the$VersionNumber, I'll probably just tweak the loader to recognizeMathicsand disable the version check for that case.
Indeed, I think that the best way to import packages that looks at a certain version is to prepare wrappers for each package. One problem right now is that we do not have a "Notebook frontend" that has the support of the WMA graphics frontend. I think that the idea for having it is to define a Pymathics module that supports all the Notebook/Cells symbols, and there, define interfaces to IWolfram and Django's frontends.
Getting closer, but not quite there yet :)
In[1]:= stream = StringToStream["(* ::Package:: *)"];
In[2]:= out = Read[stream, Hold[Expression]]
Read::readt: Invalid input found when reading (* ::Package:: *) from InputSteam[System`String, 16].
Out[2]= $Failed
Indeed, I think that the best way to import packages that looks at a certain version is to prepare wrappers for each package. One problem right now is that we do not have a "Notebook frontend" that has the support of the WMA graphics frontend. I think that the idea for having it is to define a Pymathics module that supports all the Notebook/Cells symbols, and there, define interfaces to IWolfram and Django's frontends.
The necessity of the notebook front end certainly depends a lot on the package, but in the special case of FeynCalc the front end is not necessary. All our unit tests (including examples for calculating Feynman diagrams) actually run without it.
Regarding the first, OK, it seems it will require more work...
About the second, in that case, we can add a Symbol System`$Notebooks set to False.
Yes, a System`$Notebooks symbol set to something is certainly needed.
Once FeynCalc can be properly loaded, I could also start to implement workarounds that cater for Mathics.
A short question aside: Can I somehow enable history when running mathics in the terminal? Like when you run math, you can at least use up and down arrows to scroll through the previously entered commands (also from previous sessions).
Indeed, both mathics and mathicsscript support command history. The last one also support command history between sessions.
A little bit of information about mathics and mathicsscript. The former is what you get with Mathics core. It is intended to be minimal in dependencies and requirements, because Mathics core might imported and used from front ends like Django or Jupyter or Sage. Or Mathics core might be used as a library. In these situations, the mathics CLI is superfluous.
On the other hand mathicsscript uses GNU Readline's history, input shorthand and command-completion mechanism and each of these involves extra coding. It isn't likely to be imported from other frontends like Mathics core is.
Also, in contrast to mathics, mathicsscript is intended to support Unicode and ANSI terminal escape sequences (e.g. for color) and Pygments styling. It also has limited ability to use matplotlib for networkx graphics.
Although mathics is limited, it loads faster and I often use it for quick and dirty sessions.
However whenever I need to a serious command-oriented terminal I use mathicsscript. Right now for command-line interaction using Emacs key bindings, it is better than the Django interface.
Things may change though when as the Jupyter front end progresses.
This is the current status:
In[5]:= Get["KnotTheory`", Trace->False]
Loading KnotTheory` version of September 6, 2014, 13:37:37.2841.
Read more at http://katlas.org/wiki/KnotTheory.
SetDelayed::write: Tag Condition in BR[TorusKnot[m_, n_]] /; m > 0 && n > 0 is Protected.
SetDelayed::write: Tag Condition in PD[TorusKnot[m_, n_]] /; m > 0 && n > 0 is Protected.
SetDelayed::write: Tag Condition in Crossings[TorusKnot[m_, n_]] /; m > 0 && n > 0 is Protected.
Set::write: Tag Condition in PositiveQ[X[i_, j_, k_, l_]] /; i == j || k == l || j - l == 1 || l - j > 1 is Protected.
Set::write: Tag Condition in PositiveQ[X[i_, j_, k_, l_]] /; i == l || j == k || l - j == 1 || j - l > 1 is Protected.
Set::write: Tag Condition in NegativeQ[X[i_, j_, k_, l_]] /; i == j || k == l || j - l == 1 || l - j > 1 is Protected.
Set::write: Tag Condition in NegativeQ[X[i_, j_, k_, l_]] /; i == l || j == k || l - j == 1 || j - l > 1 is Protected.
SetDelayed::write: Tag Condition in BraidLength[Knot[n_Integer, k_Integer]] /; 0 <= n <= 10 && 1 <= k <= NumberOfKnots[n] is Protected.
SetDelayed::write: Tag Condition in AllKnots[n_] /; n <= 10 is Protected.
SetDelayed::write: Tag Condition in AllKnots[n_] /; 11 <= n <= 16 is Protected.
SetDelayed::write: Tag Condition in AllKnots[n_, t_] /; 11 <= n <= 16 is Protected.
SetDelayed::write: Tag Condition in AllKnots[n_, Alternating] /; n <= 10 is Protected.
SetDelayed::write: Tag Condition in AllKnots[n_, NonAlternating] /; n <= 10 is Protected.
SetDelayed::write: Tag Condition in AllLinks[n_] /; 2 <= n <= 12 is Protected.
SetDelayed::write: Tag Condition in AllLinks[n_, t_] /; 2 <= n <= 12 is Protected.
SetDelayed::write: Tag Condition in DTCode[Knot[n_, t_, k_]] /; n <= 11 is Protected.
SetDelayed::write: Tag Condition in DT4Knots[n_, t_] /; 12 <= n <= 16 is Protected.
SetDelayed::write: Tag Condition in DTCode[Knot[n_, t_, k_]] /; 12 <= n <= 16 is Protected.
SetDelayed::write: Tag Condition in NameString[Knot[n_Integer ? (#1 <= 10&), k_Integer]] /; k <= NumberOfKnots[n] is Protected.
SetDelayed::write: Tag Condition in NameString[Knot[n_Integer ? (#1 >= 11&), Alternating, k_Integer]] /; k <= NumberOfKnots[n, Alternating] is Protected.
SetDelayed::write: Tag Condition in NameString[Knot[n_Integer ? (#1 >= 11&), NonAlternating, k_Integer]] /; k <= NumberOfKnots[n, NonAlternating] is Protected.
SetDelayed::write: Tag Condition in NameString[Link[n_Integer, Alternating, k_Integer]] /; k <= NumberOfLinks[n, Alternating] is Protected.
SetDelayed::write: Tag Condition in NameString[Link[n_Integer, NonAlternating, k_Integer]] /; k <= NumberOfLinks[n, NonAlternating] is Protected.
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
SetDelayed::write: Tag Condition in Knot[S_String ? (StringMatchQ[#1, DigitCharacter.. ~~ _ | ~~ DigitCharacter..]&)] /; (#1[[1]] <= 10 && #1[[2]] <= NumberOfKnots[#1[[1]]]&)[ToExpression /@ StringSplit[S, _ | ]] is Protected.
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
SetDelayed::write: Tag Condition in Knot[S_String ? (StringMatchQ[#1, K ~~ DigitCharacter.. ~~ a ~~ DigitCharacter..]&)] /; (#1[[1]] >= 11 && #1[[2]] <= NumberOfKnots[#1[[1]], Alternating]&)[ToExpression /@ StringSplit[S, {K, a}]] is Protected.
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
SetDelayed::write: Tag Condition in Knot[S_String ? (StringMatchQ[#1, K ~~ DigitCharacter.. ~~ n ~~ DigitCharacter..]&)] /; (#1[[1]] >= 11 && #1[[2]] <= NumberOfKnots[#1[[1]], NonAlternating]&)[ToExpression /@ StringSplit[S, {K, n}]] is Protected.
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
SetDelayed::write: Tag Condition in Knot[S_String ? (StringMatchQ[#1, L ~~ DigitCharacter.. ~~ a ~~ DigitCharacter..]&)] /; (1 <= #1[[2]] <= NumberOfLinks[#1[[1]], Alternating]&)[ToExpression /@ StringSplit[S, {L, a}]] is Protected.
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
SetDelayed::write: Tag Condition in Knot[S_String ? (StringMatchQ[#1, L ~~ DigitCharacter.. ~~ n ~~ DigitCharacter..]&)] /; (1 <= #1[[2]] <= NumberOfLinks[#1[[1]], NonAlternating]&)[ToExpression /@ StringSplit[S, {L, n}]] is Protected.
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
SetDelayed::write: Tag Condition in Knot[S_String ? (StringMatchQ[#1, DigitCharacter.. ~~ a_ ~~ DigitCharacter..]&)] /; (#1[[1]] >= 11 && #1[[2]] <= NumberOfKnots[#1[[1]], Alternating]&)[ToExpression /@ StringSplit[S, {a_}]] is Protected.
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
StringSplit::strse: String or list of strings expected at position 1 in StringSplit[S].
SetDelayed::write: Tag Condition in Knot[S_String ? (StringMatchQ[#1, DigitCharacter.. ~~ n_ ~~ DigitCharacter..]&)] /; (#1[[1]] >= 11 && #1[[2]] <= NumberOfKnots[#1[[1]], NonAlternating]&)[ToExpression /@ StringSplit[S, {n_}]] is Protected.
SetDelayed::write: Tag Condition in NextKnot[Knot[n_Integer ? (#1 <= 10&), k_Integer]] /; k < NumberOfKnots[n] is Protected.
SetDelayed::write: Tag Condition in NextKnot[Knot[n_Integer ? (#1 <= 9&), k_Integer]] /; k == NumberOfKnots[n] is Protected.
SetDelayed::write: Tag Condition in NextKnot[Knot[10, k_Integer]] /; k == NumberOfKnots[10] is Protected.
SetDelayed::write: Tag Condition in NextKnot[Knot[n_Integer ? (#1 >= 11&), t_, k_Integer]] /; k < NumberOfKnots[n, t] is Protected.
SetDelayed::write: Tag Condition in NextKnot[Knot[n_Integer ? (#1 >= 11&), Alternating, k_Integer]] /; k == NumberOfKnots[n, Alternating] is Protected.
SetDelayed::write: Tag Condition in NextKnot[Knot[n_Integer ? (#1 >= 11&), NonAlternating, k_Integer]] /; k == NumberOfKnots[n, NonAlternating] is Protected.
SetDelayed::write: Tag Condition in GaussCode[K_] /; !MatchQ[Head[K], PD | DTCode | List | String | ConwayNotation | GaussCode] is Protected.
SetDelayed::write: Tag Condition in KnotilusURL[K_] /; Head[K] =!= GaussCode is Protected.
SetDelayed::write: Tag Condition in DTCode[K_] /; !MatchQ[Head[K], DTCode | GaussCode | String] is Protected.
StringReplace::strse: String or list of strings expected at position 1 in StringReplace[ToString[poly, TeXForm], LaurentPolynomialTeXReplacementRule].
StringReplace::strse: String or list of strings expected at position 1 in StringReplace[ToString[a, TeXForm], ext{ -> extrm{].
StringReplace::strse: String or list of strings expected at position 1 in StringReplace[ToString[a, TeXForm], ext{ -> extrm{].
Syntax::sntufn: Unknown unicode longname "IndentingNewLine" (line 2729 of "./KnotTheory/init.m").
Syntax::sntxb: Expression cannot begin with "\!\(\(PowerQ[_Integer] := True;\)\[IndentingNewLine]" (line 2729 of "./KnotTheory/init.m").
@vsht Here is an approach that I found very helpful in getting Steve Skiena''s Combinatorica package working...
If the package can be subsetting to say the smallest usable piece we can focus on just that and fix Mathics bugs and add features for that. Then move on to something larger and more complex.
You are in a better position though to figure out how to subset the package so it can be more easily be worked on and debugged.
Thanks.
This
str="$Abbreviations::usage =\n\"$Abbreviations is a list of string substitution rules used when \\\ngenerating names for storing intermediate results. \\\nIt is used by OneLoop and PaVeReduce.\\\nThe elements of the list should be of the form \\\"name\\\" -> \\\"abbreviation\\\".\";"
einput = Read[StringToStream[str], Hold[Expression]]
leads to a freeze.
Also, the output of Mma for
Read[StringToStream["(* ::Package:: *)"], Hold[Expression]]
is Hold[Null] instead of {} (mathics).
@rocky Sure, but I'm afraid that the FeynCalc loader is an essential part, since this is where the definitions from all the .m files are parsed and converted to expressions. If we could get past that stage, it would become much easier to focus on a subset of functions that kind of work (but require fixes, adjustments).
BTW, is it also possible to implement new Mathics functions in Wolfram Language? I mean, Mathematica also has a lot of high level functions implemented in WL, while only some performance-critical things are written in C. That would perhaps help to add the missing functionality faster as compared to rebuilding every single Mma routine in Python.
BTW, is it also possible to implement new Mathics functions in Wolfram Language? I mean, Mathematica also has a lot of high level functions implemented in WL, while only some performance-critical things are written in C. That would perhaps help to add the missing functionality faster as compared to rebuilding every single Mma routine in Python.
I suppose anything is possible. But keep in mind it defeats the spirit of Mathics which is to provide an open-source libre free version of the Wolfram Language.
Maybe there is some way to combine the two for those that have Wolfram Language, but I personally don't have this and won't be installing it. So I can't help here in any specific way. I also am not sure of what mechanisms there are for interacting between the two.
This
str="$Abbreviations::usage =\n\"$Abbreviations is a list of string substitution rules used when \\\ngenerating names for storing intermediate results. \\\nIt is used by OneLoop and PaVeReduce.\\\nThe elements of the list should be of the form \\\"name\\\" -> \\\"abbreviation\\\".\";" einput = Read[StringToStream[str], Hold[Expression]]leads to a freeze.
OK. I will try to fix this
Also, the output of Mma for
Read[StringToStream["(* ::Package:: *)"], Hold[Expression]]is
Hold[Null]instead of{}(mathics).
OK too. This not seems too hard...
@rocky Sure, but I'm afraid that the FeynCalc loader is an essential part, since this is where the definitions from all the .m files are parsed and converted to expressions. If we could get past that stage, it would become much easier to focus on a subset of functions that kind of work (but require fixes, adjustments).
I think that this will not too hard, and the loading problems seem to be related to a reduced subset of functions, and to do this worth a lot because will help to support many other WL modules (CellsToTeX, KnotTheory, etc)
BTW, is it also possible to implement new Mathics functions in Wolfram Language? I mean, Mathematica also has a lot of high level functions implemented in WL, while only some performance-critical things are written in C. That would perhaps help to add the missing functionality faster as compared to rebuilding every single Mma routine in Python.
Indeed, you can define these functions in .m files. Then, the modules can be loaded by hand, or by placing them in the autoload folder.
Sorry, I'm afraid that you misunderstood my statement. My point was that since Mathics already provides an implementation of WL, one could add new functions to Mathics written in WL instead of Python. Of course, the WL code would still be run by Mathics, not Mathematica
For example, if I want to reimplement a function foo from Mma in Mathics, I could write foo in WL
and add that to the Mathics source. Then foo would become available in Mathics without the effort
of implementing it in pure Python. As I said, this wouldn't require the user to have Mathematica or any
other proprietary stuff installed, since Mathics is a WL interpreter: After the core of Mathics is loaded, it could just load my WL code for foo as well.
Indeed, you can define these functions in .m files. Then, the modules can be loaded by hand, or by placing them in the autoload folder.
I see, thanks! Where can I find the documentation for the autoload folder?
But I guess that for contributions to the source code you guys still prefer Python, right? Currently, in the repo there are probably no Mathics functions written in pure WL, I guess.
My point was that since Mathics already provides an implementation of WL, one could add new functions to Mathics written in WL instead of Python. Of course, the WL code would still be run by Mathics, not Mathematica
Sorry I misunderstood. By all means, please do extend mathics in WL source code.
Currently Needs[] is implemented and there are some packages you get with Mathics core: https://github.com/mathics/Mathics/tree/master/mathics/packages
More should be added. For example see https://github.com/mathics/Mathics/issues/1130 which suggests supporting individual WL language levels this way.
~~Down the line we can build off this mechanism so that mandatory packages for the basic functioning are loaded without needing Needs and I suppose we can jigger the namespace so that these appear as system builtins.~~
I forgot about autoloads. See @mmatera comment below.
As for documenting this, when I get a chance I'll add it to the developer docs: https://mathics-development-guide.readthedocs.io/en/latest/developing-code/contributing-developing.html
But as someone fresh to this feel free to add information and sections such as as we bring up here.
The repository for the developer docs is https://github.com/Mathics3/mathics-development-guide
Indeed, you can define these functions in .m files. Then, the modules can be loaded by hand, or by placing them in the autoload folder.
The autoload folder is in /mathics/mathics/autoload. The code that you drop there is loaded at the beginning.
I see, thanks! Where can I find the documentation for the autoload folder?
But I guess that for contributions to the source code you guys still prefer Python, right?
Not necessarily. We are working on improving the interpreter because otherwise, WL modules can not work. But if you have ideas about how to implement some missing symbols in WL, go ahead. It will be useful until a more efficient lower-level implementation come up.
Currently, in the repo there are probably no Mathics functions written in pure WL, I guess.
My point was that since Mathics already provides an implementation of WL, one could add new functions to Mathics written in WL instead of Python. Of course, the WL code would still be run by Mathics, not Mathematica
Sorry I misunderstood. By all means, please do extend mathics in WL source code.
Currently
Needs[]is implemented and there are some packages you get with Mathics core: https://github.com/mathics/Mathics/tree/master/mathics/packagesMore should be added. For example see #1130 which suggests supporting individual WL language levels this way.
Down the line we can build off this mechanism so that mandatory packages for the basic functioning are loaded without needing Needs and I suppose we can jigger the namespace so that these appear as system builtins.
In principle, if you define symbols in the System` context, from the point of view of the interpreter they will be treated equally than the builtin symbols.
Ok, thanks.
I noticed that e.g. as of now Collect is missing. Unfortunately CoefficientArrays is missing as well, but
at least there is CoefficientRules. What is more unfortunate, is that there is no Series.
But well, those are not so fundamental at the moment. Let us see if those issues with Read can be sorted out somehow
and then we can slowly proceed further.
Indeed, those symbols are a priority for the near future. However, both Collect and Series could be implemented in WL using the current set of instructions. For example, a basic implementation of Series can be written in pure WL with something like
Series[f_,{x_,a_,n_}] := Module[{k},SeriesData[x,a,1/k! Table[ (D[f,{x,k}]/.x->a),{k,0,n}],0,n,1]]
and then define the corresponding Normal[SeriesData[...]] and MakeBoxes[SeriesData[...]].
Afterwards, Series can be replaced by something that uses sympy, but the work in Normal and MakeBoxes can be reutilized.
@vsht For Collect[] and Series[] @mmatera was ahead of you in noticing that; #1193 and #1194 were created to track this.
Although adding these is straightforward, since there seem to be sympy equivalents particular combinations of parameters, I agree that we should folcus on getting something minimally working and then expand from there.
The idea is that if someone with expertise in WL but not in python, has the will to contribute with #1193 and #1194, could do that with code in WL. Afterward, a part of the code can be replaced by "low-level" Python builtins, but most of the work would be still reused.
Many thanks. It seems like Mathics can now parse usage messages via Read, but there is still an issue with explicit \n characters
as opposed to empty lines. For example, this works as intended
str = "foo::usage = \"foo is ...\";
bar::usage = \"bar is ...\";"
strm = StringToStream[str];
Read[strm, Hold[Expression]]
Read[strm, Hold[Expression]]
but this fails
str = "foo::usage = \n\"foo is ...\";
bar::usage = \"bar is ...\";"
strm = StringToStream[str];
Read[strm, Hold[Expression]]
Read[strm, Hold[Expression]]
To give you a better test case, what I'm currently trying to achieve is to have the attached file FCMain.m
parsed via the following code
SetDirectory[NotebookDirectory[]]
myFileAsString = Import["FCMain.m", "Text"];
strm = StringToStream[myFileAsString];
moreLines = True;
While[moreLines, einput = Read[strm, Hold[Expression]];
(*Print[einput];*)
ReleaseHold[einput];
If[einput === $Failed || MatchQ[einput, Hold[_End]],
moreLines = False]];
Notice that here the reader parses only symbol definitions and stops when it reaches the first End (line 361). Once the reader is done, the symbols are kind of declared but not defined, so that one can use e.g. ?$Abbreviations or ?$AL.
The definitions come after this step when one just loads all relevant .m files via Get/ToExpression.
Just let me know when you think that it's time to test things again. The last thing I tried was the #KnotTheoryFixes branch with your commits from the #fixReadExpressions branch on top of it. This seems to get stuck already when parsing the commented boilerplate
Read::readt: Invalid input found when reading (* ::Package:: *) from InputSteam[System`String, 302].
I found an issue with the function handling Derivatives. I will be working on that, which at some point is related to this too (we need Derivative, Series and Collect in order to make this packages work and do something interesting).
All right.
Although I'd like to comment that Series and Collect are not so crucial for FeynCalc (at least as far as simpler calculations
are concerned. In fact, instead of Collect we use our own implementation, Collect2. It relies on CoefficientArrays, though. However, one could achieve the same using Coefficient only, except that it would be slower then with CoefficientArrays. But if we get to that point, I could surely add an if condition to switch to Coefficient when using Mathics.
Update: at this point, it seems that the FeynCalc package fails to load because if fails to download the package from github: It tries to get the package doing
URLSave["https://github.com/FeynCalc/feyncalc",CreateTemporary[]]
but then, instead of the folder, it gets the webpage. Later, I will try to do what is stated in https://github.com/FeynCalc/feyncalc/wiki/Installation#st_offline_installation to see if with this at least some part of the packages loads...
Do you mean the automatic installer? But that one is pretty fragile even with genuine Mma, so I wouldn't really spend time on that. Downloading the tarball should be sufficient for everyone.
To me it looks like Read[StringToStream["..."], Hold[Expression]] still doesn't work properly, at least using the latest commit from the KnotTheoryFixes branch. Or should I try another branch?