Ground source heat pump (GSHP) systems have been identified as promising solutions to help buildings achieve future carbon neutral targets. However, two main barriers decelerate the wide adoption of this technology in every location. One barrier is the high initial cost, which is sometimes justified by high local energy prices. The other potential barrier that always imposes a major bottleneck on a transition to electrification by using GSHP systems is the impact on the electrical grid at peak demand times. In this study, a solution has been proposed to address both issues by integrating thermal and battery energy storage systems into a conventional GSHP system. Numerical simulations are performed to evaluate the technical and economic feasibility of the proposed configuration. In addition, a comparative analysis is performed with other commonly recognized GSHP configurations with and without thermal energy storage. Results show that the new configuration requires a 22% shorter borehole by using the thermal energy storage, and reduces the annual peak electricity demand by almost 500kW cumulatively by using the battery energy storage. These benefits contribute to improving the net present value of the new configuration by 23% compared to the conventional GSHP configuration.