Zika virus (ZIKV) is a mosquito-transmitted flavivirus that caused a public health emergency in the Americas when an outbreak in Brazil became linked to congenital microcephaly. Understanding how ZIKV could evade the innate immune defenses of the mother, placenta, and fetus has become central to determining how the virus can traffic into the fetal brain. ZIKV, like other flaviviruses, evades host innate immune responses by leveraging viral proteins and other processes that occur during viral replication to allow spread to the placenta. Within the placenta, there are diverse cell types with coreceptors for ZIKV entry, creating an opportunity for the virus to establish a reservoir for replication and infect the fetus. The fetal brain is vulnerable to ZIKV, particularly during the first trimester, when it is beginning a dynamic process, to form highly complex and specialized regions orchestrated by neuroprogenitor cells. In this review, we provide a conceptual framework to understand the different routes for viral trafficking into the fetal brain and the eye, which are most likely to occur early and later in pregnancy. Based on the injury profile in human and nonhuman primates, ZIKV entry into the fetal brain likely occurs across both the blood/cerebrospinal fluid barrier in the choroid plexus and the blood/brain barrier. ZIKV can also enter the eye by trafficking across the blood/retinal barrier. Ultimately, the efficient escape of innate immune defenses by ZIKV is a key factor leading to viral infection. However, the host immune response against ZIKV can lead to injury and perturbations in developmental programs that drive cellular division, migration, and brain growth. The combined effect of innate immune evasion to facilitate viral propagation and the maternal/placental/fetal immune response to control the infection will determine the extent to which ZIKV can injure the fetal brain.