This was bugging me so I hit up IMDB and did a little research lol.
Youngest daughter Judy was there for the first four seasons, her last appearance being the finale of S4. She was gone with no explanation as of the S5 premiere. Aunt Rachel (Harriet's sister) and baby Ritchie were there from the beginning, but they aged the baby up to a ~6 yrs old right before Judy's mysterious disappearance. Aunt Rachel would also be demoted to recurring character, with Eddie and Laura's friends Waldo, Maxine, and Mayra becoming regulars in the years to follow.
Despite lots of trivia entries, IMDB offered no reason (real world or behind the scenes) for actress Jaimee Foxworth's departure. Super odd that users/fans haven't provided any info either!