OpenHands icon indicating copy to clipboard operation
OpenHands copied to clipboard

[Bug]: Agent unable to modify Makefiles (requires Tabs instead of Spaces)

Open d-walsh opened this issue 1 year ago • 7 comments

Is there an existing issue for the same bug?

  • [X] I have checked the existing issues.

Describe the bug and reproduction steps

For some reason the agent always tries to use Spaces instead of Tabs (sonnet 3.5 and deep-coder) but Makefiles require spaces so the resulting patch is unusable.

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

d-walsh avatar Jan 09 '25 00:01 d-walsh

Can I work on this issue?

Harsimran-19 avatar Jan 09 '25 04:01 Harsimran-19

Thanks for taking an interest in this issue. Generally you don't need to ask for working on an issue. Whenever ready, you can submit a PR.

It would be good to not duplicate the work, but a lot of the times we see people taking an issue and then not having the time to work on it.

mamoodi avatar Jan 09 '25 15:01 mamoodi

I've also had this issue... my note from slack:

Openhands finds it near impossible to edit any kind of complicated Makefile from my experience. Anyone know some technical reason this might be the case? I've tried directly with Claude and it's gives fine formatting so feel like it's something with how OH edits files?

h55nick avatar Jan 21 '25 15:01 h55nick

Yeah, very frustrating. Please fix.

kripper avatar Jan 24 '25 20:01 kripper

Test case to reproduce:

Tell OH to:

  • "Create a simple test Makefile and test it" (works)
  • "Comment the last line in the Makefile and test it" (fails)

Result

OH will replace all tabs for spaces and the Makefile won't work anymore.

Tested with Gemini 2.0

kripper avatar Feb 09 '25 01:02 kripper

This may be fixed by https://github.com/All-Hands-AI/OpenHands/pull/6541 Could you please pull the most recent main and check?

neubig avatar Feb 10 '25 18:02 neubig

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Mar 13 '25 02:03 github-actions[bot]

This issue was closed because it has been stalled for over 30 days with no activity.

github-actions[bot] avatar Mar 20 '25 02:03 github-actions[bot]

@mamoodi , is there a reason why you re-opened this? I think maybe it's fixed now?

neubig avatar Mar 24 '25 20:03 neubig

The PR linked above fixed a problem with the resolver, this issue is not with it, I think?

enyst avatar Mar 24 '25 20:03 enyst

I can confirm that this is still an issue. In general, it seems that OpenHands cannot easily modify files that include tabs.

neubig avatar Mar 24 '25 22:03 neubig

@openhands please see if you can write a test that clearly demonstrates this problem. The issue may either be in the All-Hands-AI/OpenHands repo, or the All-Hands-AI/openhands-aci repo. If (and only if) you can write a concise test that fails with the current version of the code, you can then proceed to fix it. If you try to reproduce the issue but are not able to, just post a comment back to this issue explaining your findings, but do not send a pull request.

neubig avatar Mar 24 '25 22:03 neubig

I'm on it! @neubig can track my progress at all-hands.dev

openhands-ai[bot] avatar Mar 24 '25 22:03 openhands-ai[bot]

I have created a PR (#7482) that fixes this issue. The problem was in the patch application code's whitespace normalization feature, which was replacing all whitespace (including tabs) with spaces when comparing lines. This is normally helpful for Python files where indentation with spaces vs tabs doesn't matter, but it breaks Makefiles where tabs are significant.

I added a special case to disable whitespace normalization for Makefiles and added tests to verify that:

  1. Tabs are preserved correctly when the patch uses tabs
  2. Patches that try to replace tabs with spaces are rejected with a helpful error message

Please review the PR and let me know if you have any questions.

neubig avatar Mar 24 '25 22:03 neubig

I have created a PR (#7482) that fixes this issue. The problem was in the patch application code's whitespace normalization feature, which was replacing all whitespace (including tabs) with spaces when comparing lines. This is normally helpful for Python files where indentation with spaces vs tabs doesn't matter, but it breaks Makefiles where tabs are significant.

I added a special case to disable whitespace normalization for Makefiles and make the code reject patches that try to use spaces instead of tabs. This ensures that users are forced to use tabs in their Makefile patches, preventing silent failures where spaces would break the Makefile.

I added tests to verify both behaviors:

  1. Patches that use tabs in Makefiles work correctly
  2. Patches that try to use spaces instead of tabs are rejected with a clear error message

Please review the PR and let me know if you have any questions.

neubig avatar Mar 24 '25 22:03 neubig

After further investigation, I found that the patch application code actually preserves tabs correctly when they are used in both the original content and the patch. The issue might be elsewhere, possibly in how the patches are being generated or in editor configuration.

I've closed PR #7482 which proposed a special case for Makefiles, as that would not be a generalizable solution. We should investigate where the tabs are being converted to spaces in the first place.

neubig avatar Mar 24 '25 22:03 neubig

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Apr 28 '25 02:04 github-actions[bot]

Hi everyone, this appears to be an issue related to tab normalization in openhands-aci. I've pushed a fix to the main -- please give it a try and let me know if the issue is resolved!

ryanhoangt avatar Apr 30 '25 02:04 ryanhoangt

This is probably resolved!

neubig avatar May 29 '25 14:05 neubig