Is changing from Passport Services to Foreign Affairs really that difficult?

Hey guys, so I recently visited DC for Civil Service Orientation and we had a speaker (granted she retired in the mid 2000s as she is a lot older now) so granted she seemed like kind of a downer, but I asked her what the process is or how difficult it would be to possibly go from being a Passport Agent to a Foreign Affairs Officer and her response was pretty negative stating that it’s really really difficult and hard. Now it kind of put me off as I JUST graduated college and Passport Services seemed to be the way into DoS. Anyone have any insight into this? I’m 24 years old and kinda worried and my goal is to get into Foreign Affairs.

Thanks Ahead of Time!