Absolutely nothing, except the price. A pastry brush will cost $10 or more at a kitchen supply store, a trim brush costs about a dollar at the local hardware store.
A traditional pastry brush has a wood handle and natural or synthetic bristles, a paint brush has a wood handle and natural or synthetic bristles.
Need a sanitary pastry brush?
Take a new paint brush and rinse it in a mild bleach solution (10 parts water to one part bleach) or other sanitizer.
When it gets dirty and nasty, throw it out (and you won't feel nearly so bad about replacing it often when you've only spent $1).
There you go. Buy lots, and use them for everything from brushing glaze on cookies and pastries, to egg wash on dough, to grease on pans.