Selectivity to orientations of edges is seen at the earliest stages of visual processing in retinal orientation-selective ganglion cells (OSGCs), which are thought to prefer vertical or horizontal orientation. However, because stationary edges are projected on the hemispherical retina as lines of longitude or latitude, how edge orientation is encoded and decoded by the brain is unknown. Here, by mapping the orientation selectivity (OS) of thousands of OSGCs at known retinal locations in mice, we identify three OSGC types whose preferences match two longitudinal fields and a fourth type matching two latitudinal fields, with the members of each field pair being non-orthogonal. A geometric decoder reveals that two OS sensors yield optimal orientation decoding when approaching the deviation from orthogonality we observe for OSGC field pairs. Retinotopically organized decoding generates type-specific variation in decoding efficiency across the visual field. OS tuning is greater in the dorsal retina, possibly reflecting an evolutionary adaptation to an environmental gradient of edges.