Reduce memory by using reflection + string
I did not make a pull request as it may not be wanted but in short this is the idea:
- every time when working with big AstNode (like SyntaxTree as a result of parsing of a big file) many strings are duplicated
So at the end of: SyntaxTree СSharpParser.Parse(ITextSource program, string fileName, int initialLine, int initialColumn) method I added:
var minimiseAstMemory = new MinimiseAstTreeMemory(unit);
minimiseAstMemory.Process();
And the minimiseMemory class does the following:
public class MinimiseAstTreeMemory
{
private readonly List<AstNode> _nodes;
public MinimiseAstTreeMemory(AstNode node)
{
_nodes = new List<AstNode> {node};
}
public MinimiseAstTreeMemory(IEnumerable<AstNode> node)
{
_nodes = new List<AstNode>(node);
}
static readonly Dictionary<Type, FieldInfo[]> _fieldTypesMapping = new Dictionary<Type, FieldInfo[]>();
private void MinimizeAstNodeMemory(AstNode astNode, Dictionary<string, string> mergedTexts)
{
var astNodeType = astNode.GetType();
FieldInfo[] stringFields;
if(!_fieldTypesMapping.TryGetValue(astNodeType, out stringFields))
{
stringFields = astNodeType.GetFields(BindingFlags.Instance|BindingFlags.Public|BindingFlags.NonPublic)
.Where(fi=>fi.FieldType==typeof(string))
.ToArray();
_fieldTypesMapping[astNodeType] = stringFields;
}
foreach (var stringField in stringFields)
{
var itemText = stringField.GetValue(astNode).ToString();
string replaceText;
if (!mergedTexts.TryGetValue(itemText, out replaceText))
{
mergedTexts[itemText] = itemText;
}
else
{
itemText = replaceText;
}
stringField.SetValue(astNode, itemText);
}
if (!astNode.HasChildren)
return;
foreach (var child in astNode.Children)
{
MinimizeAstNodeMemory(child,mergedTexts);
}
}
public void Process()
{
var mergedTexts = new Dictionary<string, string>();
foreach (var astNode in _nodes)
{
MinimizeAstNodeMemory(astNode, mergedTexts);
}
}
}
This code reduces a lot big files that are to be parsed in XamarinStudio.
A way to test it: Try open NR solution and open: cs-parser.cs file.
On my machine with this patch, the memory reduction is of 20 MB comparing loading cs-parser.cs file.
Warning: this code is reflection based an also takes into account that strings are the merging og data.
Similarly, it allows (yet is not enabled) that IDEs to give all parse trees to have shared string instances, so for a multi-tab IDE, it can use the other constructor so memory can be greatly reduced.
If needed: (as I don't do the final commit, but who knows): I give the rights to do what you want with this code.
NB. If you want me to make a pull request, I can also make it but just to know if is useful.