IndexError when searching for fields in files with no newline at the of the file

[dwiel]

In order to reproduce:

x.py:

from .test import system

system('hello world')

test/__init__.py

from os import system

note that test/__init__.py must not have a newline at the end of the file. Try to find_occurrences on system in the call in x.py and you will get this IndexError:

  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/contrib/findit.py", line 35, in find_occurrences
    return _find_locations(finder, resources, job_set)
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/contrib/findit.py", line 111, in _find_locations
    for occurrence in finder.find_occurrences(resource):
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/refactor/occurrences.py", line 75, in find_occurrences
    result = filter(occurrence)
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/refactor/occurrences.py", line 205, in __call__
    if same_pyname(self.pyname, occurrence.get_pyname()):
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/utils.py", line 11, in _wrapper
    setattr(self, name, func(self, *args, **kwds))
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/refactor/occurrences.py", line 131, in get_pyname
    return self.tools.name_finder.get_pyname_at(self.offset)
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/evaluate.py", line 76, in get_pyname_at
    return self.get_primary_and_pyname_at(offset)[1]
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/evaluate.py", line 108, in get_primary_and_pyname_at
    if self.worder.is_from_aliased(offset):
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/worder.py", line 66, in is_from_aliased
    return self.code_finder.is_from_aliased(offset)
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/worder.py", line 370, in is_from_aliased
    as_start = self._find_word_start(as_end)
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/worder.py", line 131, in _find_word_start
    while current_offset >= 0 and self._is_id_char(current_offset):
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/worder.py", line 163, in _is_id_char
    return self.code[offset].isalnum() or self.code[offset] == '_'
IndexError: string index out of range

I tried to go through and find the right place to fix it, but wasn't sure where to make the change. We don't want to change the FIle.read to enforce newline at end of file. I'm having trouble getting a test case that has this error.

Making this change seems to help:

rope/base/worder.py:162
    def _is_id_char(self, offset):
+        if offset >= len(self.code):                                                                                                                                                                                                                                                      
+            return False                                                                                                                                                                                                                                                                  
        return self.code[offset].isalnum() or self.code[offset] == '_'

I'm not sure if this is the right way to go about fixing it though, it seems like it might need to be fixed somewhere else.

[mcepl]

@aligrudi ... can I get your wisdom here, please?

[aligrudi]

Zach Dwiel [email protected] wrote:

 File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/worder.py", line 131, in _find_word_start
    while current_offset >= 0 and self._is_id_char(current_offset):
  File "/home/ubuntu/anaconda/lib/python2.7/site-packages/rope/base/worder.py", line 163, in _is_id_char
    return self.code[offset].isalnum() or self.code[offset] == '_'
IndexError: string index out of range

I tried to go through and find the right place to fix it, but wasn't sure where to make the change. We don't want to change the FIle.read to enforce newline at end of file. I'm having trouble getting a test case that has this error.

You may add a test case to WordRangeFinderTest in codeanalyzetest.py.

Making this change seems to help:

rope/base/worder.py:162
    def _is_id_char(self, offset):
+        if offset >= len(self.code):
+            return False
        return self.code[offset].isalnum() or self.code[offset] == '_'

I'm not sure if this is the right way to go about fixing it though, it seems like it might need to be fixed somewhere else.

The problem is that in is_from_aliased(), (end + 1) is passed to _find_word_end() to find the end of the next word (end is len(self.code) - 1 and end + 1 goes past self.code). I wonder if is_from_aliased() function should return False if an IndexError is raised (it already does so for ValueError). Does doing so fix the problem?

Ali