Enrich Your Future 40: Why Passive Investing Gives You Back What Wall Street Steals

Listen on
Apple | Listen Notes | Spotify | YouTube | Other
Quick take
In this episode of Enrich Your Future, Andrew and Larry Swedroe discuss Larry’s new book, Enrich Your Future: The Keys to Successful Investing. In this series, they discuss Chapter 40: The Big Rocks.
LEARNING: Passive investing will give you the freedom you need.
“Indexing and passive investing have the ‘disadvantage’ of being boring. I admit it. However, if anyone needs to get their excitement in life from investing, I’d suggest they might want to consider getting another life.”
Larry Swedroe
In this episode of Enrich Your Future, Andrew and Larry Swedroe discuss Larry’s new book, Enrich Your Future: The Keys to Successful Investing. The book is a collection of stories that Larry has developed over 30 years as the head of financial and economic research at Buckingham Wealth Partners to help investors. You can learn more about Larry’s Worst Investment Ever story on Ep645: Beware of Idiosyncratic Risks.
Larry deeply understands the world of academic research and investing, especially risk. Today, Andrew and Larry discuss Chapter 40: The Big Rocks.
Chapter 40: The Big Rocks
In Chapter 40, Larry explains why passive (systematic) investing is the winning strategy in life as well as investing.
Like all the other chapters in the book, this one begins with a story used as an analogy to help understand a financial issue. In this one, a time-management expert fills a mason jar with large rocks. “Full?” she asks. The class agrees. She adds gravel, sand, and water – each filling the spaces between. When a student suggests the lesson is about fitting more into busy schedules, she corrects them:
“If you don’t put the big rocks in first, they’ll never fit at all.”
The investor’s jar
Larry explains the metaphor’s profound implication for wealth:
- Big rocks = Family, health, growth, legacy
- Gravel = Stock charts, earnings analysis
- Sand = Financial news, market commentary
- Water = Trading forums, portfolio tinkering
Larry explains that active investors start with gravel and sand, leaving insufficient time for the big rocks. They spend much of their precious leisure time watching the latest business news, studying the latest charts, scanning and posting on Internet investment discussion boards, reading financial trade publications and newsletters, and so on. Their jars fill with noise, leaving no room for life’s essentials.
Passive investors, on the other hand, ignore the ”noise” (the sand, the gravel, and the water) and place big rocks first. Their strategy operates quietly, driven by low-cost index funds and disciplined rebalancing. The result? Their jars hold what truly enriches life, giving them a sense of freedom and independence.
Two stories, one lesson
1. The physician’s regret
During the 1990s bull market, a doctor would spend nights analyzing stocks after 12-hour shifts. He turned $10,000 into $100,000 – but his marriage was on the verge of collapse. His wife no longer had a husband; his child lost a parent to the glow of stock charts. When the tech bubble burst, the money vanished.
The wake-up call was brutal: He had traded first steps and bedtime stories for digits on a screen. After reading Larry’s book, he switched to passive investing, which helped him salvage both his finances and his family. Now, he was playing the winners’ game in life and investing.
2. The executive’s discovery
A Wharton MBA and corporate treasurer spent decades analyzing stocks after work. Upon adopting passive investing, he calculated a shocking truth: He wasted 6.5 weeks per year on futile research.
Worse, this “gravel” wasn’t neutral – trading fees, taxes, and behavioral errors eroded returns. By eliminating the noise, he reclaimed 500+ annual hours for family and passions.
Why boring is the bravest choice
Larry notes that indexing and passive investing have the ‘disadvantage’ of being boring. However, he continues, investing was never meant to be exciting despite what Wall Street and the financial media want you to believe. Investing is supposed to be about achieving your financial goals with the least amount of risk.
Making the ‘boring’ choice in investing can actually be empowering, as it puts you in control and builds confidence in your financial future. Larry further explains that indexing, and passive investing in general, not only allows you to earn market returns in a low-cost and tax-efficient manner but also frees you from spending any time at all watching CNBC and reading financial publications that are essentially no more than what Jane Bryant Quinn called “investment porn.”
Play a winner’s game
If you find that you need excitement from your investments, consider setting up a separate “entertainment” account. The assets inside that account should not exceed more than a few percent of your total portfolio. Invest the rest of your assets in what I believe to be the winner’s game.
Further reading
- Paul Samuelson, Quoted in Jonathan Burton, Investment Titans (McGraw-Hill, 2001).
Did you miss out on the previous chapters? Check them out:
Part I: How Markets Work: How Security Prices are Determined and Why It’s So Difficult to Outperform
- Enrich Your Future 01: The Determinants of the Risk and Return of Stocks and Bonds
- Enrich Your Future 02: How Markets Set Prices
- Enrich Your Future 03: Persistence of Performance: Athletes Versus Investment Managers
- Enrich Your Future 04: Why Is Persistent Outperformance So Hard to Find?
- Enrich Your Future 05: Great Companies Do Not Make High-Return Investments
- Enrich Your Future 06: Market Efficiency and the Case of Pete Rose
- Enrich Your Future 07: The Value of Security Analysis
- Enrich Your Future 08: High Economic Growth Doesn’t Always Mean High Stock Market Return
- Enrich Your Future 09: The Fed Model and the Money Illusion
Part II: Strategic Portfolio Decisions
- Enrich Your Future 10: You Won’t Beat the Market Even the Best Funds Don’t
- Enrich Your Future 11: Long-Term Outperformance Is Not Always Evidence of Skill
- Enrich Your Future 12: When Confronted With a Loser’s Game Do Not Play
- Enrich Your Future 13: Past Performance Is Not a Predictor of Future Performance
- Enrich Your Future 14: Stocks Are Risky No Matter How Long the Horizon
- Enrich Your Future 15: Individual Stocks Are Riskier Than You Believe
- Enrich Your Future 16: The Estimated Return Is Not Inevitable
- Enrich Your Future 17: Take a Portfolio Approach to Your Investments
- Enrich Your Future 18: Build a Portfolio That Can Withstand the Black Swans
- Enrich Your Future 19: The Gold Illusion: Why Investing in Gold May Not Be Safe
- Enrich Your Future 20: Passive Investing Is the Key to Prudent Wealth Management
Part III: Behavioral Finance: We Have Met the Enemy and He Is Us
- Enrich Your Future 21: Think You Can Beat the Market? Think Again
- Enrich Your Future 22: Some Risks Are Not Worth Taking
- Enrich Your Future 23: Seeing Through the Frame: Making Better Investment Decisions
- Enrich Your Future 24: Why Smart People Do Dumb Things
- Enrich Your Future 25: Stock Crashes Happen—Be Prepared
- Enrich Your Future 26: Should You Invest Now or Spread It Out?
- Enrich Your Future 27: Pascal’s Wager: Betting on Consequences Over Probabilities
- Enrich Your Future 28 & 29: How to Outsmart Your Investing Biases
- Enrich Your Future 30: The Hidden Cost of Chasing Dividend Stocks
- Enrich Your Future 31: Risk vs. Uncertainty: The Investor’s Blind Spot
Part IV: Playing the Winner’s Game in Life and Investing
- Enrich Your Future 32: Trying to Beat the Market Is a Fool’s Errand
- Enrich Your Future 33: The Market Doesn’t Care How Smart You Are
- Enrich Your Future 34: Embrace the Bear: Why Market Crashes Are Your Silent Ally
- Enrich Your Future 35: Market Gurus Are Just Expensive Entertainers
- Enrich Your Future 36: The Madness of Crowded Trades
- Enrich Your Future 37 & 38: The Calendar Is a Crook & Hot Funds Are a Trap
- Enrich Your Future 39: More Wealth Does Not Give You More Happiness
About Larry Swedroe
Larry Swedroe was head of financial and economic research at Buckingham Wealth Partners. Since joining the firm in 1996, Larry has spent his time, talent, and energy educating investors on the benefits of evidence-based investing with an enthusiasm few can match.
Larry was among the first authors to publish a book that explained the science of investing in layman’s terms, “The Only Guide to a Winning Investment Strategy You’ll Ever Need.” He has authored or co-authored 18 books.
Larry’s dedication to helping others has made him a sought-after national speaker. He has made appearances on national television on various outlets.
Larry is a prolific writer, regularly contributing to multiple outlets, including AlphaArchitect, Advisor Perspectives, and Wealth Management.
Andrew Stotz 00:01
Fellow risk takers, this is your worst podcast host. In fact, I just saw some person, or a couple of people, published an episode of the show called the worst, and so I had to somewhat one of my friends and former guests, Frank, basically put a note. Wait a minute. You can't be the worst. Andrew is the worst. So the fight didn't last long. I triumphed. I'm still the worst. Andrew Stotz, from a Stotz Academy, continuing my discussion with Larry swedroe, who for three decades was the head of Research at Buckingham wealth partners. You can learn more about his story in Episode 645, Larry stands out because he bridges both the academic research world and practical investing. Now today we're diving into a chapter from his recent book, enrich your future, the keys to successful investing. And specifically we're talking about chapter 40 the big rocks, Larry, take it away.
Larry Swedroe 00:58
Yeah, thank you, Andrew, good to be back. So like all the other chapters in the book, this one begins with a story that we use to as an analogy to help understand a financial issue in this one, it's a story I read about where a professor in time management talked to a bunch of doctoral students, and she brings out a beaker and fills it up with some big rocks. And then when she couldn't get any more big rocks in, she asked the class, is the jar filled? And some couple of people shout out, yes. And she says, Oh, really. She pulls out from under the desk a bunch of smaller rocks, and starts to fill in the spaces between the big rocks until no more small rocks could fit in. And she then asks the class, is the jar now fill? And the class kind of says, not really. And so she says, That's right. She pulls out a box of sand and pours the sand in and sprinkles it until no more sand could fill and then she asked one more time, is the jar now filled? And the class says no, and says, that's good. And she pulls out a glass of water, fills it up until no more water would fill in. She then asked the class, so what's the moral of the story? And one bright, eager student jumps up and says, no matter how busy your calendar is, you can always fit in one more meeting after the laughter dies down. Says, that's not really the message, of course, the message is, if you don't put the big rocks in first, you could never put them in right and so she said, the question then is, what are the big rocks in your life? And is it trying to outperform the market or spending time with your family, et cetera. And we looked at all of the evidence. And if you put in all this time trying to beat the market, trying to outperform, professionals who are spending all of their time have a lot more access to data, as well as all of the training, et cetera, the odds are your app perform them are asymptotically close to zero. They're not zero. But especially after taxes, it's so low that it's not prudent to try. And as my Bucha co published, or co authored with Andrew burkin, The Incredible Shrinking Alpha shows it's getting harder and harder to outperform over time, it was about 20% of professionals. Professionals were outperforming back in 1998 when I wrote my first book about 13 years later, it was down to 2% even before taxes and 1% after, and it's gotten even harder ever since. So that raises the question, you know, what are the big rocks? What's important? And so I told the story in the book, which is a true story happened with me. It happened a bit after my first book was published in 98 I got a call from a doctor who relayed this story to me. It was in the late 90s, of course, and a lot of his friends doctors and doctors, as it turns out, if you ask most investment advisors, they will tell you that doctors are the worst investors because they confuse intelligence with wisdom, and because they're obviously a highly intelligent individuals. You can't get through med school and not be intelligent, then you could apply that intelligence and make you know, outperform the market. Whenever I'm I hear that. From a doctor, I always tell them, Well, I graduated number one from my MBA program. Would you let me operate on your patients? And they laugh, and I said, Well, why do you think you can use your intelligence to outperform when you have no training and investment I bet you haven't even take a single course in capital markets theory, and you get a duh kind of answer, right? So the doctor went on to tell me the story that a bunch of his friends had been making a lot of money, outperforming the market, buying these stocks that related to medicine they were researching. So he decided he ought to do it. So after coming in, after putting in maybe an 80 hour week, he'd come home go to his computer instead of sitting down to a dinner with his wife, read a book to his new baby, you know, it's new child, and spend hours researching, looking at the charts, listen to videos and all this stuff, reading blogs. And very quickly, it turned a small investment, he said, into 100 grand. The problem was he no longer had a wife, no longer was a father to his child. You know, no one got divorced at this point, but he was spending no time with his family, and his family was not happy about it. The good news was he actually, over the next several months, lost the entire 100 grand. And if someone who had read my book recommended he read my first book, the only God you'll ever need to winning investment strategy, he read it and figured out why he was playing the losers game, not the winners game, and he was one of those losers, both in investing and in the game of life, because he was focusing on the little rocks, the sand, the water and the pebbles, not the important things, his wife, his family, etc. So he after reading my book, he gave all that up, and now we had much more time to spend with the family, so now he said he was playing the winners game in life and investing. And of course, that made my day. No authors generally get rich from writing books on investing, unless you're Warren Buffett, he's already rich, or John Bogle or somebody like that. And then there was one other story I added, because it really resonated. I was introduced to a fellow who was the assistant treasurer of Anheuser Busch and in charge of their pension investments, and they were actively picking money managers in each asset class, reviewing performance and shifting every few years based on past performance. And I went and explained to him our strategy and showed him the evidence that that strategy didn't work. Past performance is not a good predictor, and it all resonated with him, and he decided to become a client of my firm. Months later, I met with him, and we're talking about how things are going. His wife was with him. Said, never mind how things are going from the investment side. Rick was his name is now spending. We count. She counted. It was 50 hours a week that, you know, he was spending, you know, sorry, 10 hours a week he was spending, previously on investing. Now, when he came home, couple hours a night during the week and stuff that he no longer was doing, he was now spending with his wife. So you multiply that I was 500 hours a year he had recaptured to spend on big rocks instead of reading barons and watching Wall Street week and reading the journal or whatever, which is really nothing more than the sand the gravel and no small pebble. So that, again, made my day, because helping somebody improve the quality of life is much more important. I think that's really the big rocks in our lives.
Andrew Stotz 09:08
There's that. There's a couple things you mentioned it where you said one line, luckily he lost all his profits within a few months, which, on the face of it, you would say, is not lucky, but it woke him up that he got back his family and his relationships.
Larry Swedroe 09:22
I have another story related to that, which is worth repeating, because this fellow eventually came to work for Buckingham, I'll tell you the story, and then he left and formed his own firm. And now the firm is called Hill investment groups, or anyone wants to go look it up. It's a large RIA on its own. And Rick Hill, someone had given him my book. He was working at some brokerage firm or something like that, and he read my book and said, Hey, this is the right way to invest. He came and applied. For a job we had just started out. We gave him a really low level job. That's all we had open at the time. But he would take anything just to get and to play the right game. And over lunch one day, he told me he had bought some high tech stock. Now this is like 99 All right, maybe it might have been early, 2000 somewhere around there. And I said to him, the worst thing that can happen to you is you'll make money on this, because then you'll confuse luck and skill. The best thing that can happen to you is it'll crash and you'll stop doing that. So I said, at the very least, set a stop loss, so if it does crash, you don't lose all of your money. Sure enough, as it turned out, lucky for him, the stock crashed. That may have been around March 2000 or somewhere around there, stock crashed, and he stopped picking stocks. Never bought another individual one, as far as I know, and then went on to a career as an advisor and formed his own firm, etc. So he found the winners game in both life and investing for his clients.
Andrew Stotz 11:12
There's another quote which is great in this, which is you quote from Paul Samuelson, and he said you shouldn't spend much time on your investments. They will that will just tempt you to pull up your plants and see how the roots are doing, and that's bad for the roots.
11:30
I love that. One
Andrew Stotz 11:32
of my favorite for sure. And then final pieces of advice out of this, I think really, really can help people who really are inclined to trade, and that is, you say, if you find that you need excitement from your investments, you should set up a special entertainment account. The assets inside that account should not exceed more than a few percent of your total portfolio. Invest the rest of your money in what I believe to be the winner's game. And I think that that is actually a very good solution for people who do feel like I got a trade,
Larry Swedroe 12:04
yeah. So you know, there are people who know they shouldn't take their IRA count to the racetrack or the casinos in Las Vegas or Macau, wherever they are, right? So, but they're willing to go and take $100 or 500 or 1000 consider an entertainment. If they blow it, they blow it, and the odds are they'll likely will blow it, because the odds are stacked against them, and they make the mistake when they're ahead, they keep playing, because it's playing with the house's money when it's their money. And you could quit at any time, but the house counts on it, and when they're behind, even though the odds, you know they'll lose more money if they play, they're trying to break even, so they just playing whether they're ahead or behind, and the more you play, the more the odds favor the house, right, right? So, but they go to the racetrack anyway, even though the house takes 17% or so in the US, and they buy lottery tickets where the house, the state and US takes 50% of the money. But that's okay. They understand the entertainment. They go to Casino. Maybe they enjoy watching people. They get the thrill of drawing to an inside street or whatever. But they don't lose their investment retirement money.
Andrew Stotz 13:25
Yeah, as we speak, my best friend Dale, who runs our coffee business, is in Cleveland, Ohio, in the one hotel that is right next to or has a casino in it, and he's there, he took a limited amount of money, and he's having a blast and but he takes a limited amount of money and he he's pretty convinced that he leaves a winner. You know, most of the time I
Larry Swedroe 13:47
should make sure he keeps a diary and actually write it down. I used to go to the casinos in Lake Tahoe. This is now 50 years ago, and when you had $2 tables at blackjack, and I would play. And those days, you were often facing like two or three deck hands so you could count cards, and I would knew which hands to draw on which, and knew when to double down, because the deck was stacked with more high cards, and I could, usually my rule was the 20 bucks or two hours. So if I was up at two hours, I was walking away. And, you know, sometimes I'd lose in 20 minutes, and sometimes I'd last two hours. But I had a little bit of fun testing my skill and stuff. And you could see the dealer laughing and smiling because he knew what, and he would let me do it because, you know, I wasn't betting 1000s that, you know, yeah, in which they would have kicked me off the tables then, or broken my leg.
Andrew Stotz 14:56
And just one last thing is, you mentioned about Doctor you had restored. Worry about Doctor. And for those people who are doctors, go to Episode 746, and you can listen to me talk to James Donnelly, who wrote the book The white coat investor. And that. There it is. And
Larry Swedroe 15:11
you see, this is how I learned how you could beat the deal. So by Ed Thorpe, yeah, he's the guy who broke the bank in Las Vegas, beat
Andrew Stotz 15:21
the dealer, a winning strategy. Man, that's amazing. So
Larry Swedroe 15:26
read his book and learned how to, you know, play blackjack at least put now, of course, you got five deck hands, and they shuffle it halfway through, and, you know, it's much harder to win. And they've got spy glasses, are looking down and all this
Andrew Stotz 15:42
stuff. Um, there's a great interview by Tim Ferriss on the Tim Ferriss show of Ed Thorpe. And I really, really enjoyed that. Just hearing,
Larry Swedroe 15:49
hearing a great movie about that, what was the name of that? I can't remember. I can't remember, but it was a great story.
Andrew Stotz 15:57
Speaking of that, you gave a great recommendation to me, and I'll mention it, which was, what was it? Department? Q, yeah. And as you said, the first few Are you enjoying it? Yeah, the first few episodes, you said, it starts a little slow, and it certainly makes you think that boy, Scottish people are just mad, angry, mean people, but maybe it was just the people who are in the but, yeah. But it did take a while. But
Larry Swedroe 16:22
the story is very clever, and the lead is really good,
Andrew Stotz 16:26
yeah, and you can see that they got a lot of room to bring new episodes of that, you know, a new series like that. So that was a good one, exactly. So anyways, let's now talk about we're going to wrap up. So I appreciate you know this story. I think it was great, and this chapter helps us, and we are now next going to discuss Chapter 41 in our next episode, not right now. And that is a tale of two strategies for listeners out there who want to keep up with all that Larry is doing, follow them on Twitter or x at Larry swedroe. Follow him also on LinkedIn, but most importantly, get his sub stack. In fact, you'll get book recommendations and what was the latest one I just saw and I went, I read. I read through it. I can't remember what it was, but my I'm a senior moment, but I read every one of yours that come in on sub stacks. So I highly recommend that people do that. So this is your worst podcast host, Andrew Stotz saying, I'll see you on the upside. You.
Connect with Larry Swedroe
Andrew’s books
- How to Start Building Your Wealth Investing in the Stock Market
- My Worst Investment Ever
- 9 Valuation Mistakes and How to Avoid Them
- Transform Your Business with Dr.Deming’s 14 Points
Andrew’s online programs
- Valuation Master Class
- The Become a Better Investor Community
- How to Start Building Your Wealth Investing in the Stock Market
- Finance Made Ridiculously Simple
- FVMR Investing: Quantamental Investing Across the World
- Become a Great Presenter and Increase Your Influence
- Transform Your Business with Dr. Deming’s 14 Points
- Achieve Your Goals