Neural_Color_Transfer icon indicating copy to clipboard operation
Neural_Color_Transfer copied to clipboard

Do you get similar or the same result as the original paper?

Open lihk11 opened this issue 7 years ago • 12 comments

Hi, there~ Thanks for your implementation! I tried to run one of the notebooks(NCT_modified), but the results didn't go very well. How about yours? Or maybe i do something wrong?

lihk11 avatar Jan 25 '19 02:01 lihk11

Hello We implemented the Neural Color Transfer and modified some of parameters but we also didn't get all fine results like paper.

We were trying to fix some of codes but, our team is guessing that the patch match is somewhat working not perfectly.

Also, the processing time is so much slower than the original paper because we implemented on Pytorch not CUDA exactly.

For now some results are similar with the paper and we are not making progress further from that implementation.

If you make better results please share with us and if we made further progress, I will update on my git.

Thanks.

rassilon712 avatar Jan 28 '19 13:01 rassilon712

Thanks for your reply! Could you point out which pictures you run that reach similar result with the paper? I think we can start with them.

lihk11 avatar Jan 29 '19 02:01 lihk11

Sorry for the late reply. We had hard time finding some good images we got. Our Input Image is Input and the style image is Style

And the result from those images are here. The images are from layer 5 to 1 below. img5S img4S img3S img2S img1S

Also we are trying to change some Markdown on front with results. But it is not completely implemented so we have lot of TODO.

rassilon712 avatar Mar 13 '19 13:03 rassilon712

Thanks for your reply! I have implemented my own version, and i think i got a similar result for the original paper. My best results: image image image

As for the picture you given, here are my results(5 to 1): image image image image image

lihk11 avatar Mar 14 '19 08:03 lihk11

I just checked your version. I think the parameter "radius" of FastGuidedFilter is not set properly. It could be larger (eg. 16). I am not very sure, cuz there are a lot of differences between our implements.

lihk11 avatar Mar 14 '19 09:03 lihk11

Hi @rassilon712 , I'm currently working on the NCT model with tensorflow and keras based on your implementation. Here is output of my implementation neural_color_transfer

It's pretty well on your given picture, but when it comes to some other pictures, the output are barely satisfied.

The biggest reason that I think might be the limit of capability of VGG19. It seems require very high semantic similarity to perform color transfer.

Now I am trying to replace VGG19 with other more semantic-oriented model.

nieguangyang avatar May 20 '19 03:05 nieguangyang

Thanks for your reply! I have implemented my own version, and i think i got a similar result for the original paper. My best results: image image image

As for the picture you given, here are my results(5 to 1): image image image image image

Hi @lihk11 , I'm currently working on NCT too and my version got bad results for the pictures in the paper. I think the biggest problem is loss d instead radius of fast guided filter. I've tried loss d exactly as showed in the paper and the modified version of rassilon712, non of them got decent output for pictures in the paper, can I have a look at your implementation of loss d?

nieguangyang avatar May 20 '19 05:05 nieguangyang

Hello, sorry for late reply. Me and my co-worker are busy on doing other projects so, this NCT progress is delayed for moments. For the implementation of our code is based on version 1 of the paper. We need to change some loss functions as the version 2 of the paper changed some loss terms. Did you check the version 2 of the paper and followed their terms? Also, my work needs some tuning for GuidedFilter, even though we have had some tests.

rassilon712 avatar May 20 '19 11:05 rassilon712

Hello, sorry for late reply. Me and my co-worker are busy on doing other projects so, this NCT progress is delayed for moments. For the implementation of our code is based on version 1 of the paper. We need to change some loss functions as the version 2 of the paper changed some loss terms. Did you check the version 2 of the paper and followed their terms? Also, my work needs some tuning for GuidedFilter, even though we have had some tests.

Thanks for reply~ I think most changes in v2 paper is about the loss d. It contains a factor 4 ** (L - 1). Although it still not work well for me. Weirdly, when I change it to 4 ** (5 - L), got better result. Therefore, now it is confusing for me... Looking forward to your later improvement and hope the paper's authors can release the source code soon, there are so many tricky parts!

nieguangyang avatar May 21 '19 03:05 nieguangyang

Yes, I also think that there are some tricks in code that they didn't mention in paper. We will work further when we have enough time and if you get some remarkable results, I hope to share the results with us !

rassilon712 avatar May 21 '19 09:05 rassilon712

Sure, I will! :)

nieguangyang avatar May 21 '19 09:05 nieguangyang

urther when we have enough time and if you get some remarkable results, I hope to share the results with us !

Hi, could you please share the code you run with me? Recently I focus on color transfer task and also learn a lots from this paper. Now I want to run out the results by myself, however, I got the wrong code and I can't find out the available implement of it at present.

Ssakura-go avatar Feb 04 '22 18:02 Ssakura-go