The central nervous system is recognized as an immunoprivileged site because peripheral immune cells do not typically enter it. Microglial cells are thought to be the main immune cells in brain. However, recent reports have indicated that neurons express the key players of innate immunity, including Toll-like receptors (TLRs) and their adaptor proteins (Sarm1, Myd88, and Trif), and may produce cytokines in response to pathogen infection. In the absence of an immune challenge, neuronal TLRs can detect intrinsic danger signals and modulate neuronal morphology and function. In this article, we review the recent findings on the involvement of TLRs and Sarm1 in controlling neuronal morphogenesis and neurodegeneration. Abnormal behaviors in TLR- and Sarm1-deficient mice are also discussed.