As sysfs has a simple pattern of taking a rpm wakeref around the user
access, we can track the local reference and drop it as soon as
possible.

Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Jani Nikula <jani.nikula@intel.com>
Reviewed-by: Mika Kuoppala <mika.kuoppala@linux.intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20190114142129.24398-5-chris@chris-wilson.co.uk