Keeping the wrong credit card at the top of your wallet is like being in a bad relationship: You’re stuck giving more than you get, making the most of the terrible terms and defending your choices to your family and friends. But it can be hard to make a change, even when your card’s a flop.

“We’re all cognitively lazy,” says Brad Klontz, a certified financial planner and associate professor of financial psychology at Creighton University. “Our natural tendency is to stay exactly where we are.”

Mental inertia can cause us to make irrational decisions in all kinds of day-to-day dealings, with long-term costs in well-being. With credit cards, though, the costs are literal. About 1 in 5 cardholders have a card with fees and rewards that are not aligned with their spending habits, according to a recent study from J.D. Power. For many, sticking with the wrong credit card can be incredibly expensive.

Here are three big ways in which your brain snookers you into using the wrong credit card and what you can do about them.

What it is: The impulse to invest more resources — time, money, energy — into a situation because you’ve already made an investment and you don’t want it to “go to waste.”

How it tricks you: Suppose you can redeem your credit card miles only for flights with an airline you no longer fly with. You’ve already paid the annual fee for the year, so you feel like you should keep using your card for all of your purchases, even though the miles are now semi-worthless. In trying to “get your money’s worth,” you’re throwing good money after bad, because whether you continue to use the card or not, that fee has been paid, and you’re not getting it back. It’s a sunk cost.

“[People] make up reasons to continue to stick with the thing they’ve already invested in,” says JoNell Strough, a professor of psychology at West Virginia University.  “They say, ‘This card has a good reputation.’ Or ‘There must be some reason I paid that $95 fee.’”

Younger people are especially susceptible to this kind of thinking, according to a2008 study co-authored by Strough. “Young adults have a bias toward imagining that sticking with a bad choice is going to turn out OK,” she says.

How to overcome it: “Make decisions based on the information you have now,” Strough says. If you’re looking for more flexible rewards, try acash-back credit card. If you’re planning on carrying a balance, go with alow-interest card.

Next, ask your issuer about converting your old card to a no-annual-fee version. Generally, making this switch won’t hurt your credit score, and you’ll be able to hold onto your rewards for possible use later. But if downgrading isn’t an option, redeem your miles as soon as you can, and cancel your account afterward. Don’t pay to hold onto a card you don’t use.

» MORE: Closing a credit card? Make sure to do these 5 things

What it is: A desire to keep things as they are; an unwillingness to “rock the boat” by making changes.

How it tricks you: Say you owe a lot of money on a card with a high 30% annual percentage rate. You’d really like to do something about that debt. Maybe you’ve even considered moving it to a 0% balance transfer credit card to save on interest. But thanks to the status quo bias, you feel like you’re stuck on a hamster wheel. Making a change seems like too much effort for too little return. So when the cashier reads your total, you smile, reach for that high-interest card again and try not to think about your balances.

“We put a premium on things that are easy for us,” Klontz says.

This bias is closely related to the endowment effect, or the tendency to value what you already have more than what you don’t have, he adds. You might prefer your credit card to other cards just because it’s in your wallet.

How to overcome it: Take a mental step back and consider the pros and cons of making a change — and of not making a change. If you’re dealing with high-interest debt, the benefits of switching will probably easily outweigh the cost.

Start with the easy part: Use a lower-interest card, debit card or cash for new purchases, instead of adding to your high-interest debt. Remember, whatever rewards you might be earning don’t outweigh the interest your issuer charges. After that, try consolidating your high-interest debt on a 0% balance transfer APR card. If your credit is strong enough to get approved, the switch mightsave you hundreds of dollars in interest. Even if you don’t qualify, it’s still a good idea to make paying down your high-interest debt a priority.

What it is: The tendency to fear making a mistake more than the consequences of inaction.

How it tricks you: Maybe you’ve heard that opening a new credit card can ding your credit temporarily, so you’re worried about making a change. Or you’re worried that you’ll apply for the wrong card. Meanwhile, your friends are asking why you’re still using that crummy card you got in college in exchange for a free T-shirt.

“You don’t want to make the wrong move,” Klontz says. “And if you think of it biologically, evolutionarily, it’s making the wrong move that hurts us.”

How to overcome it: You’re smart to consider potential pitfalls, but don’t let your fears about what might happen stop you from making a switch. The downsides of a change might not be as awful as you feared. And in some cases — like when you’re carrying a high-fee, high-interest, no-rewards credit card even though you have good credit — almost any change would be for the better.

Opening credit cards can make your scores dip by a few points. But as long as you’re not about to apply for a mortgage, it shouldn’t be a game-changer. Closing a credit card is a bigger deal, but it’s usually more costly to keep paying an annual fee for a card you never use. Finally, if you’re worried about choosing the wrong card, follow this guide to get it right.

Kenny Rogers once sang, “You got to know when to hold ’em, know when to fold ’em, know when to walk away, and know when to run.”

If your current plastic isn’t giving you the best deals, it might be time to walk away to a better card. When using the wrong card is actually worsening your financial situation, though, it’s time to run. Don’t pay annual fees on unused cards or rack up large interest charges, especially if you can’t afford them in the first place.

If you’re feeling stuck, start by taking a hard look at the cards in your wallet. Compare them to the other cards on the market. Sort through your options. Then, do the rational thing.

Claire Tsosie is a staff writer at NerdWallet, a personal finance website. Email: claire@nerdwallet.com. Twitter: @ideclaire7.

The article 3 Ways Your Brain Tricks You Into Using the Wrong Credit Cards (And What You Can Do About It) originally appeared on NerdWallet.