Exciting ways to run powerBI out of memory
I’ve been working on a project that has a rather large model, composed of a couple larger datasets (260,000 rows x 10 columns) and (270,000 rows x 25 columns). So not large at all, nothing the tabular model in powerBI can’t handle. However, the model contains a lot of calculated tables, used to aggregate data in more complicated ways and then relate that data to the appropriate dimensions. Here’s what my model looks like:
Every shaded table is a calculated table. I would run into this problem where the PBIX file would only let me load smaller datasets, as it would run out of memory when I tried to load anything larger. Eventually I found out the issue was a relationship on a derived table to a dimension that was causing problems.
Amazingly, that error in the model would consume 7 gig working memory on my local machine, at which point the process would bail.
The point of this post is to bring to light the relationships on calculated tables. You’ll know that the calculation is really running your local machine out of memory when the menu box lets you know that it’s loaded all of the rows, and then fails. When you load data in the tool, there’s nothing to let you know that the relationship/calculation is actually the problem. The only way to identify where the problem lies is to remove the relationships or calculated table.