We present a novel method for reconstructing a 3D im- plicit surface from a large-scale, sparse, and noisy point cloud. Our approach builds upon the recently introduced Neural Kernel Fields (NKF) [58] representation. It enjoys similar generalization capabilities to NKF, while simulta- neously addressing its main limitations: (a) We can scale to large scenes through compactly supported kernel func- tions, which enable the use of memory-efficient sparse lin- ear solvers. (b) We are robust to noise, through a gradi- ent fitting solve. (c) We minimize training requirements, enabling us to learn from any dataset of dense oriented points, and even mix training data consisting of objects and scenes at different scales. Our method is capable of recon- structing millions of points in a few seconds, and handling very large scenes in an out-of-core fashion. We achieve state-of-the-art results on reconstruction benchmarks con- sisting of single objects (ShapeNet [5], ABC [33]), indoor scenes (ScanNet [11], Matterport3D [4]), and outdoor scenes (CARLA [16], Waymo [49]).