Do riders get to keep the yellow jersey?
The short answer is yes! Which is why riders strive so hard, especially in the initial days of Grand Tours, and specifically the Tour de France in this case, to win stages. Riders get to keep the jersey and kit provided by the race organizers for the rest of their lives. Prolific riders will often have "award" rooms where they highlight the awards and achievements they've reached over their career - and the jerseys they've won are almost always front and center.
Hence, prior to the classification jersey becoming solidified on one competitor as the race progresses, that a number of riders often vie just for one day in a jersey, simply so they can add it to their collection.
Riders will receive a jersey for each and every day they are in the lead and so some of the top competitors may end up with more than a few closets full of jerseys they've potentially won (we're looking at you Jonas Vingegaard).
The trade teams of the riders (say Jumbo Visma for Jonas Vingegaard) may also provide additional kit in the color of the classification to the rider - say yellow leg warmers, or a yellow helmet if they feel like it.
Teams have also been known to wear yellow helmets if they're leading the teams classification of the race.