High-speed asynchronous optical sampling (ASOPS) is a novel technique for ultrafast time-domain spectroscopy (TDS). It employs two mode-locked femtosecond oscillators operating at a fixed repetition frequency difference as sources of pump and probe pulses. We present a system where the 1 GHz pulse repetition frequencies of two Ti:sapphire oscillators are linked at an offset of Deltaf(R)=10 kHz. As a result, their relative time delay is repetitively ramped from zero to 1 ns within a scan time of 100 micros. Mechanical delay scanners common to conventional TDS systems are eliminated, thus systematic errors due to beam pointing instabilities and spot size variations are avoided when long time delays are scanned. Owing to the multikilohertz scan-rate, high-speed ASOPS permits data acquisition speeds impossible with conventional schemes. Within only 1 s of data acquisition time, a signal resolution of 6 x 10(-7) is achieved for optical pump-probe spectroscopy over a time-delay window of 1 ns. When applied to terahertz TDS, the same acquisition time yields high-resolution terahertz spectra with 37 dB signal-to-noise ratio under nitrogen purging of the spectrometer. Spectra with 57 dB are obtained within 2 min. A new approach to perform the offset lock between the two femtosecond oscillators in a master-slave configuration using a frequency shifter at the third harmonic of the pulse repetition frequency is employed. This approach permits an unprecedented time-delay resolution of better than 160 fs. High-speed ASOPS provides the functionality of an all-optical oscilloscope with a bandwidth in excess of 3000 GHz and with 1 GHz frequency resolution.