An increasingly popular approach to the analysis of neural data is to treat activity patterns as being constrained to and sampled from a manifold, which can be characterized by its topology. The persistent homology method identifies the type and number of holes in the manifold, thereby yielding functional information about the coding and dynamic properties of the underlying neural network. In this work, we give examples of highly nonlinear manifolds in which the persistent homology algorithm fails when it uses the Euclidean distance because it does not always yield a good approximation to the true distance distribution of a point cloud sampled from a manifold. To deal with this issue, we instead estimate the geodesic distance which is a better approximation of the true distance distribution and can therefore be used to successfully identify highly nonlinear features with persistent homology. To document the utility of the method, we utilize a toy model comprised of a circular manifold, built from orthogonal sinusoidal coordinate functions and show how the choice of metric determines the performance of the persistent homology algorithm. Furthermore, we explore the robustness of the method across different manifold properties, like the number of samples, curvature and amount of added noise. We point out strategies for interpreting its results as well as some possible pitfalls of its application. Subsequently, we apply this analysis to neural data coming from the Visual Coding-Neuropixels dataset recorded at the Allen Institute in mouse visual cortex in response to stimulation with drifting gratings. We find that different manifolds with a non-trivial topology can be seen across regions and stimulus properties. Finally, we interpret how these changes in manifold topology along with stimulus parameters and cortical region inform how the brain performs visual computation.