Don't mix .o and .amigalib.o files

Also make sure to not build both when only one is needed.