The ability to detect light over a broad spectral range is central to several technological applications in imaging, sensing, spectroscopy and communication. Graphene is a promising candidate material for ultra-broadband photodetectors, as its absorption spectrum covers the entire ultraviolet to far-infrared range. However, the responsivity of graphene-based photodetectors has so far been limited to tens of mA W(-1) (refs 5-10) due to the small optical absorption of a monolayer of carbon atoms. Integration of colloidal quantum dots in the light absorption layer can improve the responsivity of graphene photodetectors to ∼ 1 × 10(7) A W(-1) (ref. 11), but the spectral range of photodetection is reduced because light absorption occurs in the quantum dots. Here, we report an ultra-broadband photodetector design based on a graphene double-layer heterostructure. The detector is a phototransistor consisting of a pair of stacked graphene monolayers (top layer, gate; bottom layer, channel) separated by a thin tunnel barrier. Under optical illumination, photoexcited hot carriers generated in the top layer tunnel into the bottom layer, leading to a charge build-up on the gate and a strong photogating effect on the channel conductance. The devices demonstrated room-temperature photodetection from the visible to the mid-infrared range, with mid-infrared responsivity higher than 1 A W(-1), as required by most applications. These results address key challenges for broadband infrared detectors, and are promising for the development of graphene-based hot-carrier optoelectronic applications.