stream-processing-with-apache-flink icon indicating copy to clipboard operation
stream-processing-with-apache-flink copied to clipboard

unable to create topics in Redpanda

Open hectorrios opened this issue 2 years ago • 9 comments

Hi there

I started up the Docker environment and everything came up fine from what I can tell. However when I try to execute the redpanda setup script I get the following message.

unable to create topics [accounts]: unable to dial: dial tcp 127.0.0.1:9092: connect: connection refused

I also tried running the rpk cluster info from the container and received the following.

unable to request metadata: unable to dial: dial tcp 127.0.0.1:9092: connect: connection refused

I'm running on an Intel based MacBook Pro and using colima in place of Docker Desktop, if that makes a difference.

Thank you

hectorrios avatar Jun 14 '23 11:06 hectorrios

Are you running the commands from inside or outside the container? Also if you run docker ps do you see the redpanda container running?

https://www.youtube.com/watch?v=sJ4mRqHAtjI I created this video to help with the environment setup. Are you running into issues while following that?

polyzos avatar Jun 14 '23 11:06 polyzos

Thanks for the quick feedback. I ran the rpk cluster info from inside the container and the the shell script from my host. I wasn't aware of the video, let me go through that and report back. Thanks again!

hectorrios avatar Jun 14 '23 12:06 hectorrios

Hi there. I was not having any luck getting the environment up on my work machine. So on my personal Mac I installed Docker for Desktop and followed the instructions and the environment came up without an issue. I will check back on my work machine tomorrow and get some screen grabs of the errors I'm seeing. There is obviously some kind of issue if using other container runtimes.

Thanks.

hectorrios avatar Jun 14 '23 22:06 hectorrios

@hectorrios please do share some screenshots. I wanna help and unblock you asap

polyzos avatar Jun 15 '23 06:06 polyzos

Hi @polyzos

Thank you. Here is a screenshot I grabbed just now. It seems like it's not possible to elect a leader for the cluster based on the messages.

I'm using the docker runtime available from the Colima project https://github.com/abiosoft/colima as we don't have Docker for Desktop licenses here at work.

Screenshot 2023-06-15 at 11 38 29

hectorrios avatar Jun 15 '23 09:06 hectorrios

I'm not familiar with that colima, but since the cluster spin up I suspect the issue should be elsewhere. Is there any chance you have a logs directory in place that is corrupted maybe? Can you try deleting the logs directory that is created when the container starts and try rerunning to make sure there isn't issue with that?

polyzos avatar Jun 15 '23 10:06 polyzos

The log directories looked fine, but I decided to re-clone the repo and tried again to bring up the environment. Now, the environment is coming up at all and erroring out due to a file permission error when trying to bring up the Flink JM (see screenshot).

Screenshot 2023-06-15 at 12 30 54

hectorrios avatar Jun 15 '23 10:06 hectorrios

Seems like docker doesn't have permissions there. Some workarounds could be to comment the volume mounts inside the docker-compose file, add permissions to the folder with chmod 755 <folder-name>, or check the docker desktop and give permissions to that path.

polyzos avatar Jun 15 '23 10:06 polyzos

I had an issue bringing up the docker compose on Ubuntu 20.04. I tweaked the docker-compose.yaml to use a user defined network (the container name dns resolution was failing - redpanda:9092 was failing)... (I had to change the extension to .txt as I could not upload file with .yaml extension) I was able to proceed after that without any issues...

docker-compose-yaml.txt

atulkhot avatar Aug 11 '23 14:08 atulkhot