tracking parsebib error for possibly broken bib file

[ggebbie]

Hi Joost, Thanks for the great ecosystem. I've been using it for 2-3 years and it's very helpful!

After updating ebib and parsebib, my header list of entries stops somewhere in the "B" category. Perhaps it is related to this parsebib error:

parsebib--@entry: [Parsebib error]: 131934, "Expected one of %s, got `%c'", ("}"), 64

I assume that my bib file has a syntax error or quirk somewhere. Is this error pointing to the location?

If this is an error in parsebib and not my bib file (which worked fine until now), I could submit my bib file as a test (however it is very large).

I am using ebib 20241205 and parsebib 20241219. Emacs 29.4.50.

[joostkremers]

Hi, I made some pretty fundamental changes to parsebib, so most likely the error is in parsebib, not your bib file.

The error does point to the location, but it's a bit cryptic. the number 131934 is the character location where parsebib ran into the error.

It would be easiest if you could send me your bib file, then I should be able to figure out what is going on and hopefully fix parsebib.

[ggebbie]

thank you for the response! I eliminated the error by trial and error. In a bib entry like

@article{Tanhua-Kortzinger-2007:CO2,
	timestamp = {2022-08-06 09:43:27 (EDT)},
	title = {{An estimate of anthropogenic CO2 inventory from decadal changes in oceanic carbon content}},
	year = {2007},
	journal = {Proceedings of the National Academy of Sciences of the United States of America},
	author = {Tanhua, Toste and K\"{o}rtzinger, Arne and Friis, Karsten and Waugh, Darryn W. and Wallace, Douglas W.R. and Wu, E},
	number = {9},
	pages = {3037--3042},
	volume = {104},
	doi = {10.1073/pnas.0606574104},
	issn = {00278424},
	pmid = {17360605},
	keywords = {Anthropogenic carbon, Marine chemistry, North Atlantic}
}

originally my custom function was making a key like: @article{Tanhua-K\ortzinger-2007:CO2 where the backslash must have been causing the parsebib error.

[ggebbie]

test.bib.gz

When going back to a longer bib file, I encountered other similar errors and didn't find a good way to track them down. Here's a long version of my bib file (gzipped) where unfortunately I see a lot of accumulated detritus over the years. I understand if this is just too messy of a file to work with.

[joostkremers]

Yes, there are a couple of strange things in the bib file, mostly backslashes that confuse parsebib. They're easy to fix:

test-fixed.bib.gz

I'm surprised, though, that the file worked before...

[ggebbie]

Thanks for going above and beyond to fix problems with my bib file that were unrelated to parsebib. Cheers! I confirm that parsebib runs without error for me with the fixed bib file.

I took a look at the history of my bib file in version control. The strange placeholders like

	publisher = {\},

have been there for a few years and I didn't notice any issue with ebib or parsebib. That being said, I'll work to keep my bib file clean and let you spend your time on more important things.

[joostkremers]

Thanks for going above and beyond to fix problems with my bib file that were unrelated to parsebib. Cheers! I confirm that parsebib runs without error for me with the fixed bib file.

No problem. :slightly_smiling_face:

I took a look at the history of my bib file in version control. The strange placeholders like

	publisher = {\},

have been there for a few years and I didn't notice any issue with ebib or parsebib.

Yes, I just tried with the old parser, and you're right, it parses such fields without issue. The new parser explicitly checks for escaped braces (i.e., \{ and \}) in field values, which is why it doesn't recognize {\} as a valid field: it assumes that the closing brace is escaped, so it keeps looking for a closing brace, which ends up causing an error.

Assuming that you use BibTeX, it seems that such fields are not a problem for BibTeX. I just tried with biblatex, however, and even though it doesn't issue a warning when parsing the file, the subsequent LaTeX run fails because the field value is copied into the .bbl file and LaTeX treats \} as an escaped closing brace. So it runs into the same problem as the new parsebib: it needs a closing brace that it cannot find.

This raises the question what I should do about this. On the one hand, there is the issue of backward compatibility: The new parser shouldn't really reject .bib files that the old parser handled just fine. On the other hand, I don't see the benefit of allowing things like {\}: what purpose would it serve?

I'm reopening this issue, because it's something that I need to think about a bit and perhaps act upon.

[ggebbie]

Yes, it makes sense to reopen the issue and think about a best solution. Here are some observations/notes.

Yes, I use bibtex due to journal requirements. No BibLaTeX although it sounds superior.

I had very few parse errors (10 of 6000) and I don't know their source. It's probably not Google Scholar or crossref because then I would have had more than 10 errors. I probably have a collaborator that introduced them. I don't see a purpose for the placeholder {\} when the entire entry can be deleted. My hope/sense is that placeholders like {\} are rare in the wild.

Because I hope/think that the problem will not be too widespread, a more explicit parsebib error statement would help those in my situation fix their bib files on their own.

[krosenmann]

Hi, I have same issue, and found that parser also has troubles with '@' sign if it used in entry key.

I have entries like that:

@misc{lelukhin@habr,
  author =	 {Артем Лелюхин},
  title =	 {Масло и водка: как мы избавляем ноутбуки от стикеров
                  },
  date =	 {05.03.2020},
  url =		 {https://habr.com/ru/company/avito/blog/491212/},
  urldate =	 {06.03.2020}
}

When replaced with lelukhin2habr parser falls on the next similarly used '@'

[joostkremers]

Hi, I have same issue, and found that parser also has troubles with '@' sign if it used in entry key.

I have entries like that:

@misc{lelukhin@habr,
  author =	 {Артем Лелюхин},
  title =	 {Масло и водка: как мы избавляем ноутбуки от стикеров
                  },
  date =	 {05.03.2020},
  url =		 {https://habr.com/ru/company/avito/blog/491212/},
  urldate =	 {06.03.2020}
}

Did this used to work before the update? Because the regular expression to read the entry key hasn't changed, so I think entry keys with @ in them weren't allowed before.

It seems BibTeX and biblatex do allow @ in keys, though, so I'll push an update to fix that.

[joostkremers]

It seems BibTeX and biblatex do allow @ in keys, though, so I'll push an update to fix that.

Done.

This still leaves the original issue open, though.

[ahmed-shariff]

I've also been running into the same issue. I ended up having to checkout 5.0 to remove this error. Seems like 2940de4 is where the error starts to show up. I'll keep digging and see if I can find where the issue comes from.

[joostkremers]

I've also been running into the same issue. I ended up having to checkout 5.0 to remove this error. Seems like 2940de4 is where the error starts to show up. I'll keep digging and see if I can find where the issue comes from.

What issue are you running into, exactly, because two issues came up in this thread.

I made a mistake when I tried to allow @ in keys, so it wasn't actually working. That should be fixed now in version 6.7. If that's the issue you're referring to, upgrading to the latest version (once it appears on Melpa) should fix it.

[ahmed-shariff]

I was getting an error saying Invalid character %c`. I was having trouble reproducing this consistantly. It had stopped complaining after I had updated some other packages. I still don't know how it stopped showing up. I'll update to the lastest and see if I am getting the same issue again.

[jcintasr]

Only to keep it written down in case somebody else has the same problem. Empty entries, exported from Zotero, also give parsebib errors.

[joostkremers]

Only to keep it written down in case somebody else has the same problem. Empty entries, exported from Zotero, also give parsebib errors.

Apologies for the late reply. Could you give an example of what you mean? What exactly does an "empty entry" look like?