Fundamentally the idea of karma is that any action of body, speech or mind has consequences for ourselves. We're accustomed to focusing our attention on the effect our actions have on others. To think about karma is to turn our attention inward, and think about what we're doing to ourselves.
At the most general level, you might say that being mindful of karma is simply heeding that everything we do creates a propensity to do more of the same. Cheating generates a habit of cheating. Gossip generates a habit of gossip. Jealousy generates a habit of jealousy. No action of body, speech, or mind is a single, isolated, finished thing. (Aside: actually they say that this not necessarily true. Actions that are completely untainted by attachment or aversion are said to generate no karma at all. They leave a person completely free. This is not something I'd know from experience -- I'm just repeating hearsay.)
So far we're comfortably in rationalist territory. Easy to see how this works, and verify it. Someone who's angry a lot lives in a world of conflict. Other people are angry back at them a lot. There are people who rarely go through a day without heated arguments and physical conflict, and if such a person tries not to be angry, they will have a much harder time than I would. The habits are ingrained, and the contexts of anger have been generated; angry relationships have been established. It's much harder to back out of anger once your actions have surrounded you with angry people; and it's much harder to bite back angry words if your ordinary habit is to spew them out. And most of all, if you have the habit of lovingly cultivating angry thoughts, feeding them, nurturing them, any passing irritation is likely to sweep you into a river of anger.
Now we come to the hurdle that most of us rationalists balk at. The Buddha taught plainly and emphatically that the karma we generate follows us from lifetime to lifetime. It's taught that most of the karma that ripens in this lifetime was planted in former lifetimes, and most of the karma we incur in this lifetime, likewise, won't ripen until future lives.
I don't believe this. Not exactly. But there's a certain plausibility about it. "Some are born to sweet delight / Some are born to sweet delight / Some are born to endless night," wrote Blake, and even at my most stubbornly anti-mystical I've always had to admit that he was just reporting the facts. We all know people whose financial luck, say, is uncanny -- or whose love-life is disastrous -- way beyond their apparent deserts. It's a theory that "saves the appearances" well. I guess when I feel called upon to make my rationalist self happy about this, I consider that I didn't cut my own cloth. What I am was largely created by historical circumstances, the wash of millions of actions before my time. And my actions, even the most trivial, will echo down the centuries. The time I answered my daughter snappishly six years ago may surface ten years from now when she scolds her son. The unfair racist suspicion I showed to a black man who was breaking into a car, twenty years ago -- because, as it turns out, he had locked his keys inside it -- has surely travelled on as a little ember of racial hostility, waiting to smite my children yet unborn and unbegot. If we take the emptiness of self seriously, the fact that "I" didn't commit the acts before my time, and that "I" won't suffer their consequences after my death, is actually meaningless. Some sentient being did them, and some sentient being will suffer the consequences, and "my" stream of consciousness is only arbitrarily and conventionally separate from theirs.