First JSoC blog

This is the first JSoC blog I am writing enlisting the contributions I have made as of now.

I have gotten 1 PR merged, which consisted of implementing some Flux agnostic Images.jl functions and some very simple adjoints, namely for (<:Colorant)(::Real...). I have to implement the adjoints for (<:Colorant)(x::T) where T, and this is a tricky part since every colorspace transform has some basic arithmetic associated with them.

Apart from that, I have implemented homography warps in my own capacity, which should be something like this - homography example

Now this should be noted that I have certainly applied the wrong homography matrix to the first image xD.

Right now I am checking the differentiability of warp and the functions it calls inside it. warp is itself mutating at some part (which is completely visible), but even apart from that, there are some weird problems I am coming across. I'll give you some examples.

julia> gradient(x->sum(round.(x)),B)
(nothing,)

julia> gradient(x->sum(round.(x;digits=15)),B)
ERROR: MethodError: no method matching iterate(::Nothing)

After having a talk in the #autodiff channel, Lyndon White told me this -

There is no rrule(::typeof(round), x; kwargs...) only @scalar_rule round(x) which generates rrule(::typeof(round), x) reference.
@scalar_rule doesn't actually accept keyword args, so fixing it would require writing a full rrule. Probably the fix is to add such support here.
Zygote when checking for if there is an rrule only checks if there are positional arguments than match, it doesn't check kwargs (even though it is aware of them). Not sure if can fix that or not, might be able to. Even if fixed would probably get a different error later though, reference.

I am currently in the process of figuring out a work around where I will be able to differentiate through the function seamlessly and my final goal (for this subpart) would be to train a homography matrix by regressing on the image (optimizing the photometric error).

23/06/2021