When interviewing for a job, the majority of Americans miss one critical step: negotiating their possible future salary. More than half, 58% of men and 61% of women, say they didn't ask for higher pay the last time they were hired, according to the Pew Research Center.

But that could mean leaving a lot of money on the table. "The thing I think people don't understand is that that salary that you walk in with compounds over time," says Nolan Church, former Google recruiter and current CEO of salary data company FairComp. That means that "as they do increases, they're anchoring on that salary." The higher you start, the higher that salary will get over time.

Here's how Church recommends going about negotiating for more pay during an interview.

'Companies begin the offer negotiation at the bottom'

To begin with, it's important to understand how companies come up with your offer in the first place.

Many companies "spend usually tens of thousands of dollars a year on compensation data," Church says. They get it from data providers with real-time, up-to-date figures about pay in their industry. "That's how they'll create bands," or salary ranges for specific roles, he says.

And when it's time to make that offer, "typically, companies begin the offer negotiation at the bottom of the band."

DON'T MISS: The ultimate guide to acing your interview and landing your dream job

Another guiding principle in how companies build their offer packages is compensation philosophy, which outlines their attitude toward compensation at large. This includes where in the spectrum of salary ranges they fall — do they pay more than most companies, the same, or less — how much paid time off they offer, etc.

Both of these elements play into their calculations.

'What is your compensation philosophy?'

After the interview process, when you get a job offer, Church recommends asking your prospective employer two questions. First, "how does this level map to your bands internally?"

When companies are setting their salary bands, they often have "levels one through 10," he says. The levels can be reflected in titles but aren't always so clear. He gives the example of the role of software engineer, which both people who've just graduated college and people who've worked for decades can have.

"Figuring out the specific level that you're at will indicate the level of seniority the company believes you have," he says, and therefore the kind of salary they think you're eligible for.

The second question he recommends asking is, "what is your compensation philosophy?" They could say something like "we pay 75th percentile," which means they pay better than 75% of companies in the industry. This will give you a sense of where in the market they anchor their offers.

'I want to join, if you can hit this number'

Once they've given you the offer and answered your questions, tell them, "I'm really excited about this offer," says Church. "Give me a couple of days to process and I'll circle back with you." Then start doing some research.

Check sites like Levels.fyi and Church's own company, FairComp, that can offer data about salary and levels. Lean into conversations in your network, too. Especially for those around you in your industry, try saying something like, "Hey, I want to have a conversation with you about compensation," says Church. "Would you be open to it?" Based on what you find, you can start to piece together what to ask for.

When you get back to your prospective employer, say something like, "I know you said this about the compensation philosophy," he says. "I'm really excited. I want to join, if you can hit this number … I will sign today."

They might get back to you and say yes, they might get back to you and offer a little bit less and they might get back to you and say no altogether.

The point is, "If you don't ask," says Church, "the answer is always no."

Want to land your dream job in 2024? Take CNBC's new online course How to Ace Your Job Interview to learn what hiring managers are really looking for, body language techniques, what to say and not to say, and the best way to talk about pay. CNBC Make It readers can save 25% with discount code 25OFF.