tl;dr: You will need the math at some point, but you can probably learn 
it as you go. Longer answer follows.

In my opinion fundamentally you do not need to know all the math, but 
you will initially see things differently from other people and have 
trouble reading papers if you don't.

If that is your case I suggest you look at actual source code instead of 
papers when it is available. This will give you a low level view of how 
algorithms work, and you will end up learning about the higher level 
concepts this way. After you know how an algorithm is implemented, read 
the corresponding paper to help you put names on the concepts.

For instance, in image processing, consider this 2D filter (convolution 
kernel):

     0 -1  0
    -1  4 -1
     0 -1  0

If you understand convolution (or even if you do not know it is called 
convolution, how a linear filter works) you can understand what this 
does (detect borders). You do not need to know it is called a Laplacian.

Sometimes, it is actually funny to see that even paper authors do not 
use the mathematical concepts, probably because they do not realize they 
are doing that in the first place. I remember reading a paper describing 
a process where you had to divide a space into bins and count things 
inside these bins and having a hard time understanding it until I 
realized it was just a way to compute a discrete approximation of 
correlation... I gave the Laplacian example above because I had been 
using it for years before learning about 2D Laplacians in class.

Now 3D geometry is slightly different because what the matrices do may 
not be obvious so I suggest you take a look at what a rotation matrix 
looks like before trying to understand source code that uses them (same 
thing for quaternions). But I started to play with 3D-related code 
before even learning what a matrix was, so I can testimony you do not 
need a lot of math education to start.