Boldin vs. Pralana vs. ProjectionLab: Chance of Success Comparison

[@victrolaman2007]

A man with 1 watch knows the time. A man with 2 is never sure 😂. Good info, keep it coming!

[@projectionlab]

Hey Rob, thanks for including us in your video! We noticed a couple of things we’d love to clarify and would be happy to walk you through how everything works. No black boxes here-just full transparency. Let us know if you're interested!

[@MillionaireLibrarian]

I'm glad to see this and I have the lifetime subscription with ya'll. I'm a happy customer.

[@hanwagu9967]

Perhaps full transparency would be to just note the "couple of things" in your comment for everyone to see? If projectionlab needs to clarify a walk through with a smart guy like Rob, then what does that say about users or potential users like me that are on a secondary education level. Seems like a pretty good OODA loop feedback to improve projection lab?

[@yayadxbdoh]

Interested as well to better understand about the delta between different solution as well ;)

[@beng32112]

Also interested in the couple of things to be included in the comments.

[@firesoon]

@@hanwagu9967 I can’t speak for ProjectionLab, but as a user, I can see that Rob wasn’t actually using the default settings like he claimed. They offered full transparency about how the tool works, but I can see why they might not want to point out his mistakes.

[@endorphinder]

Wow. My mind is blown. ALL of your videos are valuable. This one seems to have a probability of 60% or 75% or 95% of being the MOST valuable (so far). Looking forward to the answers you find.

[@SpookyEng1]

😂

[@scoobedoo5243]

Another great argument for using multiple tools. I use several tools to eliminate any strange biases of the black boxes or on my part.

[@TheBryanmauro]

Imagine paying a CFP thousands of dollars to print out a software generated reports full of Type II errors. Now you're either working a few more years unnecessarily or worse run out of money. Great job on this and let us know what you find out!!!

[@miatafunrun3078]

More like 1% AUM every year for Type II errors!

[@tharry8]

Robert - check how the Monte Carlo simulations handle future volatility. Some use historical returns for the forward projections, some use JP Morgan forward looking projections, some use vanguard forward looking projections and some maybe something else.

[@Jim_in_Portland]

Fantastic topic to cover Rob! Thank you for taking this on. I have been doing something similar myself, using Boldin and ProjectionLab, and now with a Financial Planner also using Right Capital… and I have discovered similar results as you. Namely that the outputs can often be significantly different between the tools despite efforts to run similar scenarios. Time sync issues such as you found are a problem. Differing assumptions, or exposing different variables to adjust can also be a problem. And ultimately there are black box issues as you mention in your conclusion. Once scenarios become ever n more complex - as is the case with Roth conversion explorations, it makes one wonder how much stock to put in what the tools are projecting. Thank you on behalf of ALL of us DIY’ers for running tests like this and hopefully giving us all a bit more transparency and confidence into how much we can “trust” the results these tools are displaying.

[@PaulStrizhevsky]

Thank you, Rob, this was an excellent topic to explore. The results are both surprising and concerning. The silver lining for me was to see Boldin turning out as the most pessimistic of them all, and that's the one I've been using. Hopefully that means that if Boldin projects a healthy percentage for our scenario, other programs would show an even better chance of success.

[@BangNguyen-ux4ie]

This got me thinking if you pay financial advisors thousands of dollars to do your retirement planning, their results would be different depending on which professional software they used, and they wouldn't know if other software that they didn't use would give different result. And you would believe the results they told you as the gospel. 😳

[@1234Krank1234]

Excellent timing, I have just been playing with multiple financial planning, software models and noticed the discrepancy in chance of success scores. I assumed it was a data entry error on my part. Thanks for investigating this issue!

[@erniedavis7217]

Excellent video, Rob! I love the idea of comparing these three planning tools. I look forward to learning what feedback you get from each of them and I also look forward to more comparison videos.

[@jameskirkpatrick361]

Replicated Rob's Roth IRA result in PL ($10.062M). Using a compound interest calculator (Nerd Wallet), if I compound Annually I get PL's result to the dollar ($10,062,657), excluding the $1/mo expense item. However, if I compound Monthly, the calculator returns $10,935,730, a difference of almost $900K. I couldn't find anywhere in PL where compounding schedule can be modified, so you get what you get. Guessing the compounding assumptions might vary across the different software platforms - this simple example illustrates how that can cause a nearly 10% discrepancy in an asset's future value. In fairness to PL, the information bubble in the Investment Growth Rate entry does indicate annual compounding.

[@ItsEverythingElse]

The longer the time frame and the higher the interest the more discrepancy you will have between monthly and annual compounding. But it tends to balance out on the expenses side. Also, consider that your entire portfolio is not generally compounding monthly anyway. In reality the discrepancy will be far less than the 9% in Rob's simple test.

[@Lukionest]

Rob, most of the difference between the tools probably comes from how they internally model volatility within their Monte Carlo. I'd guess that volatility is injected differently into each parameter within their black boxes for rate of return, inflation, interest rates, etc. I developed a spreadsheet that lets me see how much volatility affects returns. I can model a projected flat growth rate, such as 8%, on a starting balance, or I can use the historical total returns of the S&P 500 Index from whatever period I select. I can apply its volatility to my projected growth rate as a percentage while keeping the overall growth rate for the entire period unchanged. If I use 100% of the S&P Index volatility, that is effectively like asking, "What if over the next 30 years, the average growth is 8%, but that growth is as volatile as the S&P was during some prior 30-year period? No matter which historical 30-year period I choose, the ending balance is always lower than if the growth rate had been flat. As I dial down the volatility (i.e., 50% as volatile as the S&P), the ending balance climbs until it reaches the maximum at 0% volatility. If I dial up the volatility above 100% of the historical S&P, the ending balance will be significantly lower even when I set inflation to zero, with no withdrawals or additions to the account.

[@gregmetz5401]

excellent insights. thank you

[@JohnDunkelberg]

A phrase I use at work is "The purpose of a report is to drive decision making." From these tools, what I would want is information that informs my decisions. If I get back that I have a 75% success rate, I want to know what conditions led to the other 25%, this that is what drives the action I should take. Are those stagflation cases? equity crash? Those would lead me to different actions. That implies output that can allow introspection of the simulation cases.

[@RealTechSkills]

Great video content! I will be sure to watch your future videos on this topic. It is very important for us to understand why the chances of success vary so much.

[@danoberste8146]

[Looks into the black box... finds a magic 8-ball] "Ask again later"

[@tomgradel4999]

Modelling 101: Modelling is always dangerous when you don't have a good understanding of the black box. Thanks for embarking on the journey, but this is why I rely on spreadsheets as a primary tool and use modelling tools only to check my work.

[@InterCity134]

Modeling can lead to reliable results when done correctly - but to do that the user must understand the model; it cannot be a black box. Modeling allows you to find solutions that are counterintuitive- eg the Monte Hall Problem.

[@hanwagu9967]

Even if you have a good understanding of the black box, the outcomes may not be what you think or intended. Thanks Apple+ series Dark Matter.

[@danoberste8146]

"All models are wrong... but some are useful*" - Statistician George Box *ellipses delete several paragraphs in the Box paper, but summarize his point

[@lukemaxon]

Thanks, Rob for a great video and analysis. I like how you break things down. I've been using all four of these tools as well and have had similar thoughts, questions, and experiments. Glad to see I'm not alone :)

[@stevesipocz8155]

In Boldin, look at default setting for long term care

[@johnfarquhar4080]

I have my plan in three different calculators with Boldin being the most pessimistic. One significant difference is the expected LTC costs in the Boldin plan.

[@Mattxwill1]

@@johnfarquhar4080 This is what i simply do not understand in Boldin. The medical care costs seem absurdly high which greatly impact amount in retirement. Between my wife and I it claims nearly $700K in medical costs. $28K a year every year after retirment? LOL, i dont think so. And since i cant edit it I bailed and went to projection lab.

[@ATStrider]

Great video and comparison. We are basing our future on these tools and if they are not accurate it could drastically change our future success.

[@opnwideplease]

Boldin also predicts a much lower probability that Ohio State will win the National Championship.

[@GregoryLooney]

cruel...

[@PorscheSpeedster-kz6nc]

That’s just mean…I love it!!!❤😂😊

[@scottsnyder8691]

And that's using the optimistic scenario. 🤣

[@Bob-mm3yd]

Perfect timing for this review Rob. I purchased Pralana Online last month and want to purchase either Boldin and/or ProjectionLab to compare results. Cost isn’t the issue but it is the investment of time becoming proficient with these tools to determine if they meet one’s needs. Hopefully the developers will respond so you can continue with your informative videos! Many thanks, looking forward to more videos.

[@krishvenkataraman924]

Rob, Thanks for all your efforts in educating folks like me. I question the use case for this video. As a user of one of these retirement planning tools, why/when am I going to compare 3 different tools for the feel good (or bad) number? It may be helpful for these 3 vendors, if they've not already done this kind of comparison. What will be helpful is to create a list of specific real life use cases - Roth conversion, Withdrawal optimization to name a couple & try them out in these tools and rank them. Appreciate your effort!

[@bobby350z]

Great video Rob. I have all three tools and I agree, entering info in all of them is different and hard for some of them (takes time). And numbers come out different. Let us see what they have to say. I got Parlana only this week. I do like it as it shows all the numbers being computed. All these tools do need you spending time learning their ins and outs.

[@Mattxwill1]

I am at my wits end with these tools, and beginning to realize that maybe a financial planner is in my future as my patience has worn thin. (Lol this sounds like one of those scam intros but it's not). I like to think I'm a smart guy but there's a lot of things I don't understand and I hate when I make a simple change and then suddenly my retirement chance of success drops significantly. At least with bolden it tells you what your change's impact was, but there are things in that that just make no sense either. I currently am using projectionlab and the support is great but after using the others I can't help but to think it's not "guiding me" enough. Anyway, I love that these tools exist for us layman but I definitely have more respect for financial planners at this point.

[@kw7292]

Great topic Rob. Look forward to the follow-up. As usual, your comments are excellent.

[@Eric-wc7lx]

Great comparison test! Of course, the same variance in guidance and results would happen using various financial advisors at a much, much higher cost than using any of this software. At least I now know that Boldin is the most conservative!

[@ItsEverythingElse]

But we need to understand what makes it "conservative". Math is math. Are they applying some kind of adjustments that we don't know about?

[@johnbeeck2540]

Great video Rob! Glad you're pointing out these variances - might be a way to bound one's chance of success - use the most conservative and the most optimistic. ;-)

[@Gary-ib8dz]

This was great Thanks Rob. I was surprised they were so different

[@mstormes]

A couple of things to look into. Are the distributions at the beginning of the period or the end? Are returns applied at the beginning or the end of the prriod? Is inflation applied every month or at the end or beginning of the period? When are taxes paid? What data is used for the Monti Carlo (btw I was there last week)? Also, you need to start at the same point, not some late 2024 the other 2025, it makes a difference!

[@ellenyamamoto452]

This is great Rob, I have learned so much from your channel and appreciate you taking the time to ask these important questions! I had an advisor tell me I had a 58% chance and wanted to learn why that was so started figuring out how to project rates myself. I still have too many unknowns in my future to know my retirement spending but want my projection tool to be helpful when I posit the possibilities! 😅 It certainly would help to understand the basis for the different results.

[@mainerin_texas-gordon-9598]

Great video, can't wait for the follow-up.

[@Jean_Pecourt]

Very interesting comparison Rob. For someone like myself who is considering using risk-based guardrails (which increase/decrease withdrawals based on % success), that's some serious food for thought.

[@stevesinger4066]

Someone else mentioned this: The Boldin default for longterm healthcare default spending near end of life really impact success rate. You can pulldown the option that you never want to pay for LTC. It will warn you that over 70% of retirees will need LTC. I saw some major changes in my success, especially when I pulled down to the pessimistic Returns. This default setting that spikes in the last year or two may be in there even if you havent put in any other expenses!

[@EvilGenius007]

Wouldn't surprise me if the more pessimistic results are incorporating some version of the current CAPE ratio in their calculation. Or even if they all are, but they're using different formulas or different periods to look up that ratio.

[@tomdiblasi7840]

Rob, aren't there multiple types of Monte Carlo simulations that can be employed by the various providers? Might this explain the differences?

[@everydayadventure66]

Great video, really looking forward to hearing about how each provider responds, and learn more about why the differences show up.

[@ld5714]

This is excellent Rob and I appreciate you reaching out to them as you will get a better response than a normal user I suspect. I use Bolden and as you say, these differences are concerning. It will be helpful to gain more insight how they work and differ to help explain the dirrerent results. Larry, Central Valley, Ca.

[@Paul-GrnHil]

Excellent comparison. I think you hit on the major question near the end of the video. If they all have similar assets balances at the end of the projection , why different success values. It seems their projection algorithms are similar so I guess the question is; How do you define success?

[@professortweedypooper3549]

I am familiar with PL and it shows the number of simulations that succeed or fail, if the others do as well, how do those compare? Trying to see how if the ending balance is similar, is the number of failures driving the difference in success rates?

[@ItsEverythingElse]

The success score is basically just that, the number of simulations that succeeded versus the number that failed.

[@professortweedypooper3549]

@ True, however PL counts Large surplus and Just made it as success and Almost made it, and Failed in the middle as failures. SoI guess that rules that out as the source for the difference.

[@ItsEverythingElse]

Some software calculates things monthly and others calculate yearly and that can account for much of the different results. And I think they may handle tax rates quite differently so the differences can become even larger when taxes are starting being included. As far as Monte Carlo simulations go, the success scores might be based on different percentile ranges. They SHOULD be based on the 0-100 range but if they are not (might be 20-80 or something) then the score will be very different. Also, Boldin only uses 1000 iterations in their Montte Carlo so their result may vary a bit more from run to run. But that shouldn't necessarily make it wildly inaccurate.

[@GregoryLooney]

and even the monthly expense distributions could differ with end of the month vs beginning of the month.

[@ItsEverythingElse]

@@GregoryLooney Right. There are so many variables. Not to mention that we are projecting 30 years into the future when we don't even know what will happen next year!

[@jefflentz1630]

I didn't see any attempt that you modeled the same standard deviation in the Monte Carlo simulations between packages for your assumed 8% return. For Pralana, was it all stocks, stocks and bonds, did you use the defaults of SD=1.5*Return or enter values to match other programs. My guess, you probably haven't done due diligence to really have a good apple and apple comparisons. Posting all pertinent modeling details in description would be very helpful to replicate and follow along.

[@MichigantoFlorida]

Good evening Mr Rob, sadly enough there are so many folks who simply are clueless when it comes to finance. Your videos really help those folks. For my personal investment style I have always focused on quality and growth stocks. Today at the age of 75 I still have the same investment philosophy, keep it simple, basic and make sure you know where your money is going. Be careful with those advisors who rely on animal spirits. Keep up the good work!

[@HelenMiller-i2s]

i love watching your video it gives me motivation

[@hanwagu9967]

If Rob can't figure it out, I'm heading to my bunker.

[@stevefiete]

I agree with the comments that methods used for MC simulation are likely very different. This video inspired me to do a test in Boldin. I set up a scenario that had a single roth 401k will 1 million and there were no withdrawals or deposits, just like in the video. At the end of the timeline (37 years) the 80% confidence interval from the MC simulation was (820k, 2.1 mil). That means there is a 10% chance of getting and ending balance below 820k and a 10% chance of being above 2.1 mil.. The I created a scenario with 4 roth 401k accounts each with 250k in them. So both scenarios had the same starting balance and return assumptions. However the second scenario 80% confidence interval was (1.1 mil, 1.8 mil). This is evidence that Boldin assumes that the investment accounts returns are not 100% correlated with each other. I would like to know from Boldin what correlation assumption they make. That has a big impact on planning. Going forward I will enter balance information into consolidated accounts in Bolding. For example, I will combine both my wife's 401k and mine into a single 401k account since they have the same distribution of assets.

[@davidmclifton1]

Rob should have looked at and showed what the modeling settings are in ProjectionLab - in ProjectionLab you can use historical returns for example. Not clear to me which settings he has.

[@timomera33]

I’ve heard Moshe Milevsky say most of the companies don’t understand at all deep level how their own Monte Carlo calculators work- in terms of the derivatives and other functions they have baked in. They use “off the shelf” models that may not be applicable

[@ItsEverythingElse]

Monte Carlo simulation is very straightforward math. It mostly just comes down to how the statistical parameters are set.

[@timomera33]

@ that may have been what he was referring to. The essence being they couldn’t explain to him the parameters they chose and why.

[@timomera33]

@ Dr Milevsky mentioned it in “The Rational Reminder” podcast. Episode 122. 44;40 mark if interested to hear his take.

[@ItsEverythingElse]

Maybe I missed it but what rate of inflation did you use? I notice that ProjectionLab allows random distributions of inflation. Do the other softwares do that or do they use a fixed rate? Also what were your asset allocations? Were they the same in all softwares? Might make a difference once you start calculating taxes?

[@louismolino8674]

Thanks Rob. I have been looking forward to these comparison videos. Also look forward to the responses from each of the developers. The specific standard deviation for each asset class can be set in Pralana. I don't believe this is the case for Boldin and I don't think they disclose what is used. Also, in Pralana you set the real rate of return and inflation. I believe Boldin sets the nominal rate of return and inflation. You didn't show these, but I assume these were set to be apples to apples.

[@hanwagu9967]

Boldin does specify, "The standard deviation, while not explicitly disclosed, closely mirrors historical returns." But, couldn't your optimistic and pessimistic inputs in Boldin serve as your variable deviation?

[@jeff96762]

Great videos!! I didn't hear you talk about pricing. I have Boldin that for the $130/yr, they are GREAT!!! How about the others? Overall, how are the others with Roth Conversion analysis, scenarios, etc?

[@davidwright7083]

Data/parameters used (eg., random, historical returns ),number simulation runs used, etc in each package?

[@alealealealealealealeale]

Very interesting!

[@DPTrainor1]

Thank You.

[@davidperry2725]

Great idea, thanks. Also, the spambots are busy tonight. I must have reported 100 comments.

[@ItsEverythingElse]

Thanks, that was out of control there.

[@josephimanna9210]

Super interesting. Might it be useful to vary the hypothetical situation with different longevity dates (age 70, 75, 80, 85 ect) to see when they each start to deviate (% success and final $ amount). Might have to stress the scenario more to see a difference at earlier age of death, by reducing Roth IRA amount. Or that just might raise more questions....🙄

[@kinggeek1960]

The higher the number of successes, the higher the chance of UNDERSPENDING in retirement. You need to loop in income lab in these comparisons.

[@jameskirkpatrick361]

In Boldin, how do I create a test plan when I already have a baseline plan with account connections, etc? I tried creating a scenario off the Baseline plan, but it replicates all the connected accounts and won’t let me delete them from the scenario (along with other restrictions). PL is much more obvious, you just create a new plan. Is there a way to save a Boldin Baseline plan so I can paste it back in later?

[@jaythaxton2386]

Rob, I look forward to your videos. Thank's for all you do. Can I ask one question? My wife is 75 and still works. She has not taken RMD's yet. When must she start. Should she wait until after the new year or does it matter?

[@ashsmith9135]

Rob, Good topic, thanks for the video. Given the difference in results, did you double check and then triple check with each app vendor that you correctly entered settings that you intended? Often, user error is the explanation.

[@hanwagu9967]

Well, if you are offering a consumer product/service, should the product/service be so difficult to require 2x, 3x, 4x, 5x checking?

[@GaryOgata]

I use Boldin and know that there is a default setting for long term care expense (I saw someone mentioned this already). Wonder if other software includes default expenses.

[@michaelevans5328]

How does your PlanVision eMoney compare to these three diy apps?

[@aliciagipson5725]

Hi Rob, my name is Alicia, & I had an older sister pass away in 2022, who left me some money. I contacted the financial institution, Vanguard; I informed them at the time of her death, that I would not be doing any investing. I was told that the funds were not available to me for two years, & that I needed to be put all the money into an IRA. After listening to your program a few times am learning more about other investments options, can I use the funds that from an IRA for maximum cash flow?

[@miatafunrun3078]

This doesn't surprise me. First Monte Carlo is not standardized. A better test, would be to run the Historical Analysis test. You can do this in Pralana, not sure about the others. Second, the algorithm is different e.g. monthly withdrawals vs yearly. I trust Pralana because I can trace down exactly what is happening in the tables and validate. Software that is all bling and hides the details from you is a warning to me. I have access to Right Capital and some stuff does not compute e.g. it gets my yearly dividends wrong which is basic stuff!. Roth conversions it wants to make some asset allocation changes that I have no control over. No way would I trust that.

[@ItsEverythingElse]

Most professionals agree that Monte Carlo is a better approach. One problem with historical analysis is what do you do with the results? You're still looking for the median and the spread. What would really be cool is a tool that provides both side-by-side.

[@miatafunrun3078]

@@ItsEverythingElse I like to look at worst case years like retiring in 1966. Will I survive it and how low did my Portfolio go. I do look at Monte Carlo as well. Can't have too many sanity checks...

[@michaelsd284]

Great Video Rob. Just another reason to not rely on a single resource for planning your financial future. Looks like the Cryto Bots have found your comment page. Sad to say, but you might have to start managing your commenters/subscribers as its become pointless to sift through them for additional knowledge/insight.

[@InterCity134]

From a design of experiments basis I can guess that maybe the number of runs in the simulations could have varied between them as well as the parameters they use and the data they draw on to feed the simulations , It withstanding the equations that nodes the market. The conclusion one can draw from the set of 3 is none can be trusted , because two agreeing doesn’t mean they are more accurate. They can be equally wrong. For those projections to have any merit in interpreting they need to be open source and standardized. That way the community of investment analysts can comb over the simulation equations and inputs and assumptions and make sure all the needed stuff is there, and maybe standardize weighting system to get to a common metric - like the MPG rating on cars. Reality isn’t matching it but everyone is running the same test conditions. As it stands I don’t believe any of them.

[@go2gym]

Could you do a vid comparing Right Capital?

[@lindsaynewell6319]

as Rob mentioned, he was using a "client" version of RC (from Kevin Lum at Foundry - I have the same one) and it doesn't allow rate of return to be set by the user (it just has various pre-configured asset mixes). The full version of RC is only available as a professional advisor tool with limited functionality available to the client so it really isn't possible for Rob (or anyone that isn't a professional advisor) to do an apples to apples comparison.

[@go2gym]

@@lindsaynewell6319 .... Thank you. I hadn't watched it yet, but I use RC and saw it not included so I figured I'd request.

[@stevenmorris2736]

@Rob, have asked a few times but maybe it is getting lost. Could you please do an OnTrajectory vs Boldin please. I would appreciate it.

[@Gary-b2n]

My takeaway from this is don't try to run it down to 0

[@hanwagu9967]

Why not? If you run it to $0 at end of life, you are 100% successful and you had a 100% chance of success.

[@scoaste]

I'm sure life rarely conforms to any software projection anyway.

[@danoberste8146]

Maybe a few of the Monte Carlo assumptions is that you'll lose your nerve and sell in a down market? 🤣 Moral of the story is: Get a second opinion... and use the one you like best. 😬

[@hanwagu9967]

Isn't confirmation bias bad?

[@danoberste8146]

@@hanwagu9967 Oh, absolutely! I'm not recommending getting 2nd (3rd, 4th...) opinions to get the result you want, I'm suggesting that a LOT of people (including myself 😬) do just that. It's pernicious.

[@scr82566]

At about the 9:54 mark you should be using $96,000

[@Taicho116]

I think he is reducing the 4k per month to 3k per month for the 2nd mil in the traditional because he also sets the expenses to 7k a month in the next program not 8k. I my guess is this a quick and easy 25% taxes but it made it a bit confusing for me too.

[@gordonsteen8415]

These plans create a lot o fear, because it seems most people are projected to run out of money before we die. That ia majoe peoblem. How are the paid planners dealing with this?

[@ItsEverythingElse]

The math is what it is.

[@gregcampbell4467]

I wouldn't hold my breath waiting for a software company to pull back the curtain on their algorithms. But i can see them throwing their marketing departments at trying to sell us on the idea that theirs is high quality and the others are flawed.

[@toddbarney4738]

Just a user of Pralana for most of a year now. It is just a small handful of people, as far as I know, and they readily answer questions and explain how they get results when questioned in their forums. Their user manual is pretty detailed in explaining not only what it's doing, but also how, in many cases. It can be downloaded for free; you don't have to buy the product to examine it. You can also browse their forums to see the questions raised and how they've responded.

[@ChildofGod888]

Wow…scary. Does nothing from my confidence level. 😂. Well at least we can say that Bolden is most conservative. If everyone’s wrong, at least we’re a bit safer with Bolden.

[@bscottking]

Freakin crypto Spam is making the comments section damn near worthless