Ionic current fluctuations in graphene nanopore devices are a ubiquitous phenomenon and are responsible for degraded spatial and temporal resolution. Here, we descriptively investigate the impact of different substrate materials (Si and quartz) and membrane thicknesses on noise characteristics of graphene nanopore devices. To mitigate the membrane fluctuations and pin-hole defects, a SiNx membrane is transferred onto the substrate and a pore of approximately 70 nm in diameter is perforated prior to the graphene transfer. Comprehensive noise study reveals that the few layer graphene transferred onto the quartz substrate possesses low noise level and higher signal to noise ratio as compared to single layer graphene, without deteriorating the spatial resolution. The findings here point to improvement of graphene based nanopore devices for exciting opportunities in future single-molecule genomic screening devices.