fides
fides copied to clipboard
ethyca
Special Characters in Custom Taxonomy Entities Throw `500` Errors
Bug Description
When adding a Taxonomy Entity, if a user includes a special character or whitespace, the API returns a 500 - internal server error and presents that in the UI.
Steps to Reproduce
- Spin up a server
- Browse to http://localhost:3000/taxonomy
- Click
Add Taxonomy Entity + - In the
Data Categoryfield, enter a name that either has a special character (e.g.something_10%) or a whitespace (e.g.this is a test) - Receive
Internal Server Error - Check logs on server
Expand for full stacktrace
2023-01-06 22:45:56.900 [ERROR] (logger:_log_exception:24): FidesKeys must only contain alphanumeric characters, '.', '_' or '-'. Value provided: this is a test
Traceback (most recent call last):
File "/root/.local/lib/python3.10/site-packages/anyio/streams/memory.py", line 94, in receive
return self.receive_nowait()
│ └ <function MemoryObjectReceiveStream.receive_nowait at 0xffffb116d990>
└ MemoryObjectReceiveStream(_state=MemoryObjectStreamState(max_buffer_size=0, buffer=deque([]), open_send_channels=0, open_rece...
File "/root/.local/lib/python3.10/site-packages/anyio/streams/memory.py", line 89, in receive_nowait
raise WouldBlock
└ <class 'anyio.WouldBlock'>
anyio.WouldBlock
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 43, in call_next
message = await recv_stream.receive()
│ └ <function MemoryObjectReceiveStream.receive at 0xffffb116da20>
└ MemoryObjectReceiveStream(_state=MemoryObjectStreamState(max_buffer_size=0, buffer=deque([]), open_send_channels=0, open_rece...
File "/root/.local/lib/python3.10/site-packages/anyio/streams/memory.py", line 114, in receive
raise EndOfStream
└ <class 'anyio.EndOfStream'>
anyio.EndOfStream
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.10/multiprocessing/spawn.py", line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
│ │ └ 4
│ └ 7
└ <function _main at 0xffffb1d0ec20>
File "/usr/local/lib/python3.10/multiprocessing/spawn.py", line 129, in _main
return self._bootstrap(parent_sentinel)
│ │ └ 4
│ └ <function BaseProcess._bootstrap at 0xffffb1f35cf0>
└ <SpawnProcess name='SpawnProcess-1' parent=1 started>
File "/usr/local/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
│ └ <function BaseProcess.run at 0xffffb1f35360>
└ <SpawnProcess name='SpawnProcess-1' parent=1 started>
File "/usr/local/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
│ │ │ │ │ └ {'config': <uvicorn.config.Config object at 0xffffb1d0ae90>, 'target': <bound method Server.run of <uvicorn.server.Server obj...
│ │ │ │ └ <SpawnProcess name='SpawnProcess-1' parent=1 started>
│ │ │ └ ()
│ │ └ <SpawnProcess name='SpawnProcess-1' parent=1 started>
│ └ <function subprocess_started at 0xffffb12c3880>
└ <SpawnProcess name='SpawnProcess-1' parent=1 started>
File "/root/.local/lib/python3.10/site-packages/uvicorn/subprocess.py", line 76, in subprocess_started
target(sockets=sockets)
│ └ [<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 8080)>]
└ <bound method Server.run of <uvicorn.server.Server object at 0xffffb1d0aef0>>
File "/root/.local/lib/python3.10/site-packages/uvicorn/server.py", line 60, in run
return asyncio.run(self.serve(sockets=sockets))
│ │ │ │ └ [<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('0.0.0.0', 8080)>]
│ │ │ └ <function Server.serve at 0xffffb12c2b90>
│ │ └ <uvicorn.server.Server object at 0xffffb1d0aef0>
│ └ <function run at 0xffffb1d15d80>
└ <module 'asyncio' from '/usr/local/lib/python3.10/asyncio/__init__.py'>
File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
│ │ └ <coroutine object Server.serve at 0xffffb10d7bc0>
│ └ <method 'run_until_complete' of 'uvloop.loop.Loop' objects>
└ <uvloop.Loop running=True closed=False debug=False>
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 36, in coro
await self.app(scope, request.receive, send_stream.send)
│ │ │ │ │ │ └ <function MemoryObjectSendStream.send at 0xffffb116e170>
│ │ │ │ │ └ MemoryObjectSendStream(_state=MemoryObjectStreamState(max_buffer_size=0, buffer=deque([]), open_send_channels=1, open_receive...
│ │ │ │ └ <property object at 0xffffaf9c9170>
│ │ │ └ <starlette.requests.Request object at 0xffff843365f0>
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <starlette.middleware.cors.CORSMiddleware object at 0xffff8ccc82b0>
└ <starlette.middleware.base.BaseHTTPMiddleware object at 0xffff8ccc82e0>
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 92, in __call__
await self.simple_response(scope, receive, send, request_headers=headers)
│ │ │ │ │ └ Headers({'x-forwarded-host': 'localhost:3000', 'x-forwarded-proto': 'http', 'x-forwarded-port': '3000', 'x-forwarded-for': ':...
│ │ │ │ └ <bound method MemoryObjectSendStream.send of MemoryObjectSendStream(_state=MemoryObjectStreamState(max_buffer_size=0, buffer=...
│ │ │ └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.httptools_impl.RequestResponseCycle object at 0xffff843...
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <function CORSMiddleware.simple_response at 0xffffaec8ac20>
└ <starlette.middleware.cors.CORSMiddleware object at 0xffff8ccc82b0>
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 147, in simple_response
await self.app(scope, receive, send)
│ │ │ │ └ functools.partial(<bound method CORSMiddleware.send of <starlette.middleware.cors.CORSMiddleware object at 0xffff8ccc82b0>>, ...
│ │ │ └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.httptools_impl.RequestResponseCycle object at 0xffff843...
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <starlette.middleware.base.BaseHTTPMiddleware object at 0xffff8ccc8250>
└ <starlette.middleware.cors.CORSMiddleware object at 0xffff8ccc82b0>
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 68, in __call__
response = await self.dispatch_func(request, call_next)
│ │ │ └ <function BaseHTTPMiddleware.__call__.<locals>.call_next at 0xffff8cbad630>
│ │ └ <starlette.requests.Request object at 0xffff76e21570>
│ └ <function dispatch_log_request at 0xffff8e9b6560>
└ <starlette.middleware.base.BaseHTTPMiddleware object at 0xffff8ccc8250>
> File "/fides/src/fides/api/main.py", line 101, in dispatch_log_request
response = await call_next(request)
│ └ <starlette.requests.Request object at 0xffff76e21570>
└ <function BaseHTTPMiddleware.__call__.<locals>.call_next at 0xffff8cbad630>
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 46, in call_next
raise app_exc
└ FidesValidationError("FidesKeys must only contain alphanumeric characters, '.', '_' or '-'. Value provided: this is a test")
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 36, in coro
await self.app(scope, request.receive, send_stream.send)
│ │ │ │ │ │ └ <function MemoryObjectSendStream.send at 0xffffb116e170>
│ │ │ │ │ └ MemoryObjectSendStream(_state=MemoryObjectStreamState(max_buffer_size=0, buffer=deque([]), open_send_channels=0, open_receive...
│ │ │ │ └ <property object at 0xffffaf9c9170>
│ │ │ └ <starlette.requests.Request object at 0xffff76e21570>
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <slowapi.middleware.SlowAPIMiddleware object at 0xffff8ccabf70>
└ <starlette.middleware.base.BaseHTTPMiddleware object at 0xffff8ccc8250>
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 68, in __call__
response = await self.dispatch_func(request, call_next)
│ │ │ └ <function BaseHTTPMiddleware.__call__.<locals>.call_next at 0xffff8cbac280>
│ │ └ <starlette.requests.Request object at 0xffff76410be0>
│ └ <bound method SlowAPIMiddleware.dispatch of <slowapi.middleware.SlowAPIMiddleware object at 0xffff8ccabf70>>
└ <slowapi.middleware.SlowAPIMiddleware object at 0xffff8ccabf70>
File "/root/.local/lib/python3.10/site-packages/slowapi/middleware.py", line 56, in dispatch
response = await call_next(request)
│ └ <starlette.requests.Request object at 0xffff76410be0>
└ <function BaseHTTPMiddleware.__call__.<locals>.call_next at 0xffff8cbac280>
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 46, in call_next
raise app_exc
└ FidesValidationError("FidesKeys must only contain alphanumeric characters, '.', '_' or '-'. Value provided: this is a test")
File "/root/.local/lib/python3.10/site-packages/starlette/middleware/base.py", line 36, in coro
await self.app(scope, request.receive, send_stream.send)
│ │ │ │ │ │ └ <function MemoryObjectSendStream.send at 0xffffb116e170>
│ │ │ │ │ └ MemoryObjectSendStream(_state=MemoryObjectStreamState(max_buffer_size=0, buffer=deque([]), open_send_channels=0, open_receive...
│ │ │ │ └ <property object at 0xffffaf9c9170>
│ │ │ └ <starlette.requests.Request object at 0xffff76410be0>
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <starlette.exceptions.ExceptionMiddleware object at 0xffff8ccabf10>
└ <slowapi.middleware.SlowAPIMiddleware object at 0xffff8ccabf70>
File "/root/.local/lib/python3.10/site-packages/starlette/exceptions.py", line 93, in __call__
raise exc
File "/root/.local/lib/python3.10/site-packages/starlette/exceptions.py", line 82, in __call__
await self.app(scope, receive, sender)
│ │ │ │ └ <function ExceptionMiddleware.__call__.<locals>.sender at 0xffff765265f0>
│ │ │ └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.httptools_impl.RequestResponseCycle object at 0xffff843...
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <fastapi.middleware.asyncexitstack.AsyncExitStackMiddleware object at 0xffff8ccabee0>
└ <starlette.exceptions.ExceptionMiddleware object at 0xffff8ccabf10>
File "/root/.local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
raise e
File "/root/.local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
│ │ │ │ └ <function ExceptionMiddleware.__call__.<locals>.sender at 0xffff765265f0>
│ │ │ └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.httptools_impl.RequestResponseCycle object at 0xffff843...
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <fastapi.routing.APIRouter object at 0xffffaf5dae30>
└ <fastapi.middleware.asyncexitstack.AsyncExitStackMiddleware object at 0xffff8ccabee0>
File "/root/.local/lib/python3.10/site-packages/starlette/routing.py", line 670, in __call__
await route.handle(scope, receive, send)
│ │ │ │ └ <function ExceptionMiddleware.__call__.<locals>.sender at 0xffff765265f0>
│ │ │ └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.httptools_impl.RequestResponseCycle object at 0xffff843...
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <function Route.handle at 0xffffaf9df5b0>
└ <fastapi.routing.APIRoute object at 0xffff8e9ca500>
File "/root/.local/lib/python3.10/site-packages/starlette/routing.py", line 266, in handle
await self.app(scope, receive, send)
│ │ │ │ └ <function ExceptionMiddleware.__call__.<locals>.sender at 0xffff765265f0>
│ │ │ └ <bound method RequestResponseCycle.receive of <uvicorn.protocols.http.httptools_impl.RequestResponseCycle object at 0xffff843...
│ │ └ {'type': 'http', 'asgi': {'version': '3.0', 'spec_version': '2.3'}, 'http_version': '1.1', 'server': ('172.30.0.8', 8080), 'c...
│ └ <function request_response.<locals>.app at 0xffff8e9b6a70>
└ <fastapi.routing.APIRoute object at 0xffff8e9ca500>
File "/root/.local/lib/python3.10/site-packages/starlette/routing.py", line 65, in app
response = await func(request)
│ └ <starlette.requests.Request object at 0xffff84203a00>
└ <function get_request_handler.<locals>.app at 0xffff8e9b68c0>
File "/root/.local/lib/python3.10/site-packages/fastapi/routing.py", line 221, in app
solved_result = await solve_dependencies(
└ <function solve_dependencies at 0xffffaf59e560>
File "/root/.local/lib/python3.10/site-packages/fastapi/dependencies/utils.py", line 560, in solve_dependencies
) = await request_body_to_args( # body_params checked above
└ <function request_body_to_args at 0xffffaf59e680>
File "/root/.local/lib/python3.10/site-packages/fastapi/dependencies/utils.py", line 695, in request_body_to_args
v_, errors_ = field.validate(value, values, loc=loc)
│ │ │ │ └ ('body',)
│ │ │ └ {}
│ │ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
│ └ <function ModelField.validate at 0xffffb0b48280>
└ ModelField(name='resource', type=DataCategory, required=True)
File "/root/.local/lib/python3.10/site-packages/pydantic/fields.py", line 857, in validate
v, errors = self._validate_singleton(v, values, loc, cls)
│ │ │ │ │ │ └ None
│ │ │ │ │ └ ('body',)
│ │ │ │ └ {}
│ │ │ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
│ │ └ <function ModelField._validate_singleton at 0xffffb0b485e0>
│ └ ModelField(name='resource', type=DataCategory, required=True)
└ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
File "/root/.local/lib/python3.10/site-packages/pydantic/fields.py", line 1074, in _validate_singleton
return self._apply_validators(v, values, loc, cls, self.validators)
│ │ │ │ │ │ │ └ <member 'validators' of 'ModelField' objects>
│ │ │ │ │ │ └ ModelField(name='resource', type=DataCategory, required=True)
│ │ │ │ │ └ None
│ │ │ │ └ ('body',)
│ │ │ └ {}
│ │ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
│ └ <function ModelField._apply_validators at 0xffffb0b48700>
└ ModelField(name='resource', type=DataCategory, required=True)
File "/root/.local/lib/python3.10/site-packages/pydantic/fields.py", line 1121, in _apply_validators
v = validator(cls, v, values, self, self.model_config)
│ │ │ │ │ │ └ <member 'model_config' of 'ModelField' objects>
│ │ │ │ │ └ ModelField(name='resource', type=DataCategory, required=True)
│ │ │ │ └ ModelField(name='resource', type=DataCategory, required=True)
│ │ │ └ {}
│ │ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
│ └ None
└ <function BaseModel.validate at 0xffff8e9b6950>
File "/root/.local/lib/python3.10/site-packages/pydantic/class_validators.py", line 313, in <lambda>
return lambda cls, v, values, field, config: validator(v)
│ │ │ │ │ │ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
│ │ │ │ │ └ <bound method BaseModel.validate of <class 'fideslang.models.DataCategory'>>
│ │ │ │ └ <class 'pydantic.config.BaseConfig'>
│ │ │ └ ModelField(name='resource', type=DataCategory, required=True)
│ │ └ {}
│ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
└ None
File "/root/.local/lib/python3.10/site-packages/pydantic/main.py", line 704, in validate
return cls(**value)
│ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
└ <class 'fideslang.models.DataCategory'>
File "/root/.local/lib/python3.10/site-packages/pydantic/main.py", line 339, in __init__
values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
│ │ │ └ {'fides_key': 'this is a test', 'name': '', 'description': '', 'is_default': False}
│ │ └ <attribute '__class__' of 'object' objects>
│ └ DataCategory()
└ <function validate_model at 0xffffb0b4ba30>
File "/root/.local/lib/python3.10/site-packages/pydantic/main.py", line 1056, in validate_model
v_, errors_ = field.validate(value, values, loc=field.alias, cls=cls_)
│ │ │ │ │ │ └ <class 'fideslang.models.DataCategory'>
│ │ │ │ │ └ <member 'alias' of 'ModelField' objects>
│ │ │ │ └ ModelField(name='fides_key', type=FidesKey, required=True)
│ │ │ └ {}
│ │ └ 'this is a test'
│ └ <function ModelField.validate at 0xffffb0b48280>
└ ModelField(name='fides_key', type=FidesKey, required=True)
File "/root/.local/lib/python3.10/site-packages/pydantic/fields.py", line 857, in validate
v, errors = self._validate_singleton(v, values, loc, cls)
│ │ │ │ │ │ └ <class 'fideslang.models.DataCategory'>
│ │ │ │ │ └ 'fides_key'
│ │ │ │ └ {}
│ │ │ └ 'this is a test'
│ │ └ <function ModelField._validate_singleton at 0xffffb0b485e0>
│ └ ModelField(name='fides_key', type=FidesKey, required=True)
└ 'this is a test'
File "/root/.local/lib/python3.10/site-packages/pydantic/fields.py", line 1074, in _validate_singleton
return self._apply_validators(v, values, loc, cls, self.validators)
│ │ │ │ │ │ │ └ <member 'validators' of 'ModelField' objects>
│ │ │ │ │ │ └ ModelField(name='fides_key', type=FidesKey, required=True)
│ │ │ │ │ └ <class 'fideslang.models.DataCategory'>
│ │ │ │ └ 'fides_key'
│ │ │ └ {}
│ │ └ 'this is a test'
│ └ <function ModelField._apply_validators at 0xffffb0b48700>
└ ModelField(name='fides_key', type=FidesKey, required=True)
File "/root/.local/lib/python3.10/site-packages/pydantic/fields.py", line 1121, in _apply_validators
v = validator(cls, v, values, self, self.model_config)
│ │ │ │ │ │ └ <member 'model_config' of 'ModelField' objects>
│ │ │ │ │ └ ModelField(name='fides_key', type=FidesKey, required=True)
│ │ │ │ └ ModelField(name='fides_key', type=FidesKey, required=True)
│ │ │ └ {}
│ │ └ 'this is a test'
│ └ <class 'fideslang.models.DataCategory'>
└ <function FidesKey.validate at 0xffffafc0e680>
File "/root/.local/lib/python3.10/site-packages/pydantic/class_validators.py", line 313, in <lambda>
return lambda cls, v, values, field, config: validator(v)
│ │ │ │ │ │ └ 'this is a test'
│ │ │ │ │ └ <bound method FidesKey.validate of <class 'fideslang.validation.FidesKey'>>
│ │ │ │ └ <class 'pydantic.config.Config'>
│ │ │ └ ModelField(name='fides_key', type=FidesKey, required=True)
│ │ └ {}
│ └ 'this is a test'
└ <class 'fideslang.models.DataCategory'>
File "/root/.local/lib/python3.10/site-packages/fideslang/validation.py", line 30, in validate
raise FidesValidationError(
└ <class 'fideslang.validation.FidesValidationError'>
fideslang.validation.FidesValidationError: FidesKeys must only contain alphanumeric characters, '.', '_' or '-'. Value provided: this is a test
Expected behavior
A few things:
- Return
400including reason for error - Validate the field on the client side to prevent it from happening at all
Environment
- Version:
2.4.0 - OS: MacOS
- Python Version:
3.10.4 - Docker Version:
