linux-dmabuf: Fix crash with no valid modifiers

We shouldn't free &modifier_invalid because it wasn't allocated
with malloc()

Signed-off-by: Derek Foreman <derekf@osg.samsung.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>