Passing primitive types by reference is faster??

Tweet

If we asume references are internally implemented as pointers, then passing by reference anything whose size is equal or less than a pointer's size(4 bytes) is actually slower that passing it by value? Is that correct? for example, passing a char by reference would imply passing a pointer by value which is four times the size of a char.