In the Old Testament, the Holy Spirit was given selectively and temporarily to indwell certainly individuals for special ministries. It was not universal nor was it permanent. David’s words in Psalm 51:11 make sense in light of 1 Samuel 10:5-13, where the Spirit came mightily upon Saul, enabling him to serve as King of Israel. We know from 1 Samuel 16:14 and 18:10 that the Holy Spirit departed from Saul, and was replaced by an “evil spirit” sent by God. God was taking away Saul’s kingdom, and so too the Spirit that empowered him as king. The Spirit then came upon David (1 Samuel 16:13). We can see, then, why David would be concerned about the Spirit leaving him, as the Spirit departed from Saul. God did not take the kingdom from David, nor His Spirit.
Christ explained the difference in the ministry of the Holy Spirit in John 14:17 when He told the disciples, “but you know Him, for He dwells with you and will be in you.” The change in prepositions (“with” and “in”) and tense (present versus future) suggests a difference in the ministry of the Spirit in Old Testament times (Pentecost had not yet occurred and the church had not yet begun when the Lord spoke these words) and New Testament times when the Spirit came to indwell all believers permanently. If you note, in John 14:16, Christ said, in relation to the Spirit, “that He may be with you forever.” Then in Ephesians 4:30, Paul exhorts believers, “grieve not the Holy Spirit, with whom you were sealed for [until] the day of redemption.” The Holy Spirit is not the one doing the sealing, but the He is the seal. God the Father is the one who seals us with the Spirit (cf. 2 Cor. 1:21-22) and Paul teaches us this is “unto the day of redemption.” We are assured that all believers are indwelt by the Holy Spirit (Romans 8:9), by whom we have been baptized (1 Corinthians 12:13). We are also assured, “I will never leave you nor forsake you” (Hebrews 13:5—interestingly a citation of Deuteronomy 31:6 and Joshua 1:5). Christians may quench the Spirit and grieve the Spirit, but so far as I understand it, the Spirit never departs from us in the sense that He did Saul, and in the sense that David mentioned in Psalm 51:11.