Yes, and that's why it doesn't feel believable.
It seems like DC is going back and forth with whether Duke is part of the inner circle of batfamily or not, and it's not very consistent.
From what it looks like recently, I'd say that the immediate circle of the batfmily is the Robins and Barbra, and Duke isn't really there anymore.
Cass and steph now seem to be included more, but Duke actually feels less included in the family than them.
I'm not sure how it could be solved. Maybe letting him interact more with the batfamily in Urban legends?
Or maybe DC just want him as part of the outsiders rather than a regular batfamily character, that's also an option.