Extends the fix in f2f9a2cbaf to also
workaround permission issues noticed by people using AGP systems.
Cc: stable@vger.kernel.org # 3.16: f2f9a2c: drm/nouveau: fix regression
Cc: stable@vger.kernel.org # 3.16+
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>