PerspectiveMap question on Stackoverflow

[timholy]

https://stackoverflow.com/questions/45772848/perspective-warp-an-image-in-julia

I haven't dug into this here, but the core issue seems to be how to define a perspective transformation that takes a 2d input and returns a 2d output.

[Evizero]

Just from looking at it, I think that issue just concerns ImageTransformations. The fix to me looks to be two fold (without checking)

1.) use the new warp (change is tagged thanks to you) 2.) make sure to specify the output indices, because by default that performs inv to see where pixel end up (see https://github.com/JuliaImages/ImageTransformations.jl/blob/master/src/warp.jl#L82)

[timholy]

It's more than that:

julia> using CoordinateTransformations, ImageTransformations, Interpolations, Colors, ColorVectorSpace, TestImages

julia> img = testimage("light");

julia> M = [1 0 0; 0 1 0; -1/5 0 1]   # a 3x3 perspective transformation matrix
3×3 Array{Float64,2}:
  1.0  0.0  0.0
  0.0  1.0  0.0
 -0.2  0.0  1.0

julia> tform = PerspectiveMap() ∘ inv(LinearMap(M))
(CoordinateTransformations.PerspectiveMap() ∘ LinearMap([1.0 0.0 0.0; 0.0 1.0 0.0; 0.2 0.0 1.0]))

julia> warp(img, tform, indices(img))
ERROR: DimensionMismatch("matrix A has dimensions (3,3), vector B has length 2")
Stacktrace:
 [1] generic_matvecmul!(::Array{Float64,1}, ::Char, ::Array{Float64,2}, ::SVector{2,Int64}) at ./linalg/matmul.jl:407
 [2] warp!(::Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2}, ::Interpolations.FilledExtrapolation{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2,Interpolations.BSplineInterpolation{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2,Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2},Interpolations.BSpline{Interpolations.Linear},Interpolations.OnGrid,0},Interpolations.BSpline{Interpolations.Linear},Interpolations.OnGrid,ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}}}, ::CoordinateTransformations.ComposedTransformation{CoordinateTransformations.PerspectiveMap,CoordinateTransformations.LinearMap{Array{Float64,2}}}) at /home/tim/.julia/v0.6/ImageTransformations/src/warp.jl:89
 [3] warp(::Array{ColorTypes.RGB{FixedPointNumbers.Normed{UInt8,8}},2}, ::CoordinateTransformations.ComposedTransformation{CoordinateTransformations.PerspectiveMap,CoordinateTransformations.LinearMap{Array{Float64,2}}}, ::Tuple{Base.OneTo{Int64},Base.OneTo{Int64}}) at /home/tim/.julia/v0.6/ImageTransformations/src/warp.jl:96

julia> using StaticArrays

julia> tform(@SVector([1,1]))
ERROR: DimensionMismatch("matrix A has dimensions (3,3), vector B has length 2")
Stacktrace:
 [1] generic_matvecmul!(::Array{Float64,1}, ::Char, ::Array{Float64,2}, ::SVector{2,Int64}) at ./linalg/matmul.jl:407
 [2] (::CoordinateTransformations.ComposedTransformation{CoordinateTransformations.PerspectiveMap,CoordinateTransformations.LinearMap{Array{Float64,2}}})(::SVector{2,Int64}) at /home/tim/.julia/v0.6/CoordinateTransformations/src/core.jl:37

[timholy]

Perhaps what we need is a type that essentially appends 1 to the SVector? EDIT: this might make PerspectiveMap invertible, unlike my push solution below.

[timholy]

Aha, I didn't know about push. Here's a complete example (EDIT: slightly more polished on SO):

julia> using Images, TestImages, StaticArrays, CoordinateTransformations

julia> M = @SMatrix [1 0 0; 0 1 0; -1/1000 0 1]   # a 3x3 perspective transformation matrix
3×3 StaticArrays.SArray{Tuple{3,3},Float64,2,9}:
  1.0    0.0  0.0
  0.0    1.0  0.0
 -0.001  0.0  1.0

julia> tform = PerspectiveMap() ∘ inv(LinearMap(M))
(CoordinateTransformations.PerspectiveMap() ∘ LinearMap([1.0 0.0 0.0; -0.0 1.0 0.0; 0.001 -0.0 1.0]))

julia> tform2(x) = tform(push(x, 1))
tform2 (generic function with 1 method)

julia> img = testimage("lighthouse");

julia> imgw = warp(img, tform2, indices(img));

I can post that to the SO post, but it occurs to me that we should put this somewhere in docs. How much of that should go here versus in JuliaImages?

[IshitaTakeshi]

Seems @timholy's tform2 assumes depth=1 but I'd like to specify depth per pixel. Do you have a future plan to add the functionality like it?

[c42f]

@IshitaTakeshi I don't think we will do this, because you can get the same effect by permuting the axes with a LinearMap before passing to PerspectiveMap.

[c42f]

We could possibly consider adding more options to the cameramap convenience function to add this permutation for you.

[IshitaTakeshi]

Thanks for the answer

[yakir12]

Just posting a link to my trials and tribulations (don't worry, it has a good ending, kind of) on Discourse concerning this issue: https://discourse.julialang.org/t/help-converting-a-transformation-matrix