Don't take an extra ref on the sink and source because that creates a reference
cycle. Instead, use the invalidate method of the clock when the sink and source
are freed. This way, we don't call into the time function anymore after the
objects are disposed.