The rhizosphere microbiome is extremely important for improving soil and plant health. Though various methods have been proposed for the sampling rhizosphere microorganisms, these still lacks standardization. Molecular analysis based on 16S rRNA sequences revealed that the choice of sampling methods influences the end results of the data analysis. The first step in this process is the efficient extraction of good quality DNA with high yield and purity from the sample. Similarly, the choice of DNA extraction method plays a major role in determining the outcome of the analysis. Here we compared the conventional “pull and shake” method with our current “core” method for sampling of the rhizosphere soils. Phylogenetic analysis based on 16S rRNA showed significant differences in terms of microbial community structure in terms of beta diversity, taxonomic distribution of microbial community and ANCOM analysis while alpha diversity did not show any statistically significant differences. We also compared four different DNA extraction methods across eight contrasting soils including a rhizosphere soil sample. The result indicated that the DNA extraction method did not have a measurable effect on alpha diversity and taxonomic distribution of microbial community. However, there was a significant difference in terms of beta diversity and ANCOM analysis across DNA extraction methods. Therefore, improvements in both rhizosphere sampling and DNA extraction methods have the potential to better represent the actual rhizosphere microbiome present in the soil.