renku-python Error while creating Dataset - cli: renku dataset create -d Information [...]

Describe the bug

Hi to all, I was just trying to create a dataset, and then, this happened. It does not allow to create it, and honestly I don't know why, or what I had done wrong.

Details

Not too many details to write... Just that this has happened without me having run any other Renku command. Hence, I don't know what I messed up.

In any case, at the very end, I add the output of renku doctor, as it shows some errors.

Renku version: 1.6.0 OS: Darwin (Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000) Python: 3.9.12

Traceback

Traceback (most recent call last):
  File "[...]/site-packages/renku/infrastructure/gateway/project_gateway.py", line 34, in get_project
    return self.database_dispatcher.current_database["project"]
  File "[...]/site-packages/renku/infrastructure/database.py", line 235, in __getitem__
    return self._root[key]
KeyError: 'project'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[...]/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "[...]/site-packages/renku/core/dataset/dataset.py", line 155, in create_dataset
    project_id=client.project.id,
  File "[...]/site-packages/inject/__init__.py", line 342, in injection_wrapper
    return sync_func(*args, **kwargs)
  File "[...]/site-packages/renku/core/management/repository.py", line 174, in project
    self._project = project_gateway.get_project()
  File "[...]/site-packages/renku/infrastructure/gateway/project_gateway.py", line 36, in get_project
    raise ValueError() from e
ValueError

Additional context

This is the output of running renku doctor

Error: Exception raised when running check_dataset_old_metadata_location
	Traceback (most recent call last):
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/gateway/project_gateway.py", line 34, in get_project
	    return self.database_dispatcher.current_database["project"]
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/database.py", line 235, in __getitem__
	    return self._root[key]
	KeyError: 'project'

	The above exception was the direct cause of the following exception:

	Traceback (most recent call last):
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/core/migration/utils/__init__.py", line 189, in read_project_version
	    return client.project.version
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/inject/__init__.py", line 342, in injection_wrapper
	    return sync_func(*args, **kwargs)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/core/management/repository.py", line 174, in project
	    self._project = project_gateway.get_project()
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/gateway/project_gateway.py", line 36, in get_project
	    raise ValueError() from e
	ValueError

	During handling of the above exception, another exception occurred:

	Traceback (most recent call last):
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/command/doctor.py", line 54, in _doctor_check
	    ok, problems_ = getattr(checks, check)(client=client, fix=fix, force=force)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/command/checks/datasets.py", line 43, in check_dataset_old_metadata_location
	    old_metadata = get_pre_0_3_4_datasets_metadata(client)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/core/migration/utils/__init__.py", line 180, in get_pre_0_3_4_datasets_metadata
	    project_is_pre_0_3 = int(read_project_version(client)) < 2
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/core/migration/utils/__init__.py", line 191, in read_project_version
	    yaml_data = read_yaml(client.renku_path.joinpath(OLD_METADATA_PATH))
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/core/util/yaml.py", line 62, in read_yaml
	    with Path(path).open(mode="r") as fp:
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/pathlib.py", line 1252, in open
	    return io.open(self, mode, buffering, encoding, errors, newline,
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/pathlib.py", line 1120, in _opener
	    return self._accessor.open(self, flags, mode)
	FileNotFoundError: [Errno 2] No such file or directory: '/Users/luissalamanca/Dropbox/My_stuff/05_SDSCresearch/02_NLP/Code/democrasci-parsedpdfs/.renku/metadata.yml'

Error: Exception raised when running check_project_structure
	Traceback (most recent call last):
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/gateway/project_gateway.py", line 34, in get_project
	    return self.database_dispatcher.current_database["project"]
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/database.py", line 235, in __getitem__
	    return self._root[key]
	KeyError: 'project'

	The above exception was the direct cause of the following exception:

	Traceback (most recent call last):
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/command/doctor.py", line 54, in _doctor_check
	    ok, problems_ = getattr(checks, check)(client=client, fix=fix, force=force)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/command/checks/validate_shacl.py", line 77, in check_project_structure
	    data = ProjectSchema().dump(client.project)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/inject/__init__.py", line 342, in injection_wrapper
	    return sync_func(*args, **kwargs)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/core/management/repository.py", line 174, in project
	    self._project = project_gateway.get_project()
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/gateway/project_gateway.py", line 36, in get_project
	    raise ValueError() from e
	ValueError

Error: Exception raised when running check_project_id_group
	Traceback (most recent call last):
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/gateway/project_gateway.py", line 34, in get_project
	    return self.database_dispatcher.current_database["project"]
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/database.py", line 235, in __getitem__
	    return self._root[key]
	KeyError: 'project'

	The above exception was the direct cause of the following exception:

	Traceback (most recent call last):
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/command/doctor.py", line 54, in _doctor_check
	    ok, problems_ = getattr(checks, check)(client=client, fix=fix, force=force)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/inject/__init__.py", line 342, in injection_wrapper
	    return sync_func(*args, **kwargs)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/command/checks/project.py", line 40, in check_project_id_group
	    current_project = client.project
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/inject/__init__.py", line 342, in injection_wrapper
	    return sync_func(*args, **kwargs)
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/core/management/repository.py", line 174, in project
	    self._project = project_gateway.get_project()
	  File "/Users/luissalamanca/anaconda3/envs/demockg/lib/python3.9/site-packages/renku/infrastructure/gateway/project_gateway.py", line 36, in get_project
	    raise ValueError() from e
	ValueError

Aug 08 '22 07:08 lusamino

Simply I want to re-iterate this issue. It happened again, after having tried to clean a bit the data folder to avoid having a folder with the same name of the dataset I was going to create. But still, the same error prompt. Now, I will have to erase the repo and start again, but still, I leave the error here, in case you could identify a bug.

Details

Please verify and redact the details.

Renku version: 1.8.1 OS: Darwin (Darwin Kernel Version 21.3.0: Wed Jan 5 21:37:58 PST 2022; root:xnu-8019.80.24~20/RELEASE_ARM64_T6000) Python: 3.9.12

Traceback

Traceback (most recent call last):
  File "[...]/site-packages/renku/infrastructure/gateway/project_gateway.py", line 31, in get_project
    return project_context.database["project"]  # type: ignore
  File "[...]/site-packages/renku/infrastructure/database.py", line 235, in __getitem__
    return self._root[key]
KeyError: 'project'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[...]/site-packages/renku/ui/cli/dataset.py", line 622, in create
    create_dataset_command()
  File "[...]/site-packages/renku/command/command_builder/command.py", line 252, in execute
    output = self._operation(*args, **kwargs)  # type: ignore
  File "[...]/site-packages/renku/core/dataset/dataset.py", line 169, in create_dataset
    project_id=project_context.project.id,
  File "[...]/site-packages/renku/domain_model/project_context.py", line 165, in project
    return project_gateway.get_project()
  File "[...]/site-packages/renku/infrastructure/gateway/project_gateway.py", line 33, in get_project
    raise ValueError("Cannot get project") from e
ValueError: Cannot get project

Thanks!

Nov 01 '22 15:11 lusamino

Sorry this somehow slipped through my radar, I've added it to the next sprint. You don't by any chance still have a repo in this state that causes this problem?

Nov 18 '22 11:11 Panaetius

We couldn't reproduce this. It was likely due to the project using a preview version of renku, though we cannot be sure. I'll close this, but please reopen if it happens again and you have a repository where we can look at it.

Dec 14 '22 13:12 Panaetius