<?xml version="1.0" encoding="UTF-8" ?>
<?xml-stylesheet type="text/xsl" href="https://communities.bentley.com/cfs-file/__key/system/syndication/rss.xsl" media="screen"?><rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking</link><description /><dc:language>en-US</dc:language><generator>Telligent Community 12</generator><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking</link><pubDate>Tue, 02 May 2023 13:47:05 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Jesse Dringoli</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Current Revision posted to OpenFlows | Water Infrastructure Wiki by Jesse Dringoli on 5/2/2023 1:47:05 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1 id="mcetoc_1gorgaoka0"&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h2 id="mcetoc_1gorgaoka1"&gt;Solution - Check for excessive change tracking records&lt;/h2&gt;
&lt;p&gt;First, check if you have &lt;strong&gt;Change Tracking&lt;/strong&gt; enabled. This is located under the Review menu. Click Change Tracking and see if the box is checked next to &amp;quot;Track Changes&amp;quot;. If so, choose &amp;quot;view tracked changes&amp;quot; and check the number of records. If it is large (for example millions of rows), this can cause the database file to be very large and reduce performance especially when opening and saving the model. This could be caused by performing bulk operations such as ModelBuilder imports and constraint based design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&amp;nbsp;&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08 (and greater), you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent.&lt;/p&gt;
&lt;p&gt;In addition, a Bulk Archive features was added in the CONNECT Edition Update 4 release (10.04.00.XXX). With this, you will be able to archive a large number of items from the Change Tracking database. This will remove the items from the model database file and store the archived changes in an external database file.&lt;/p&gt;
&lt;p&gt;You can access this by going to the Review tab&lt;span&gt;, click &amp;quot;Bulk Archive.&amp;quot; You can choose how many records to keep or delete all change tracking data. After performing the Bulk Archine, it is recommended to&amp;nbsp;&lt;/span&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compact the database&lt;/a&gt;&lt;span&gt;.&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;&lt;strong&gt;Note:&lt;/strong&gt;&amp;nbsp;This does not replace the Archive feature in the Change Tracking manager.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;For more information, see the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/2806.BulkArchive1.png" /&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/1348.BulkArchive2.png" /&gt;&lt;/p&gt;
&lt;p&gt;&amp;quot;Archive remaining records&amp;quot; means it will save a .sqlite file with the change tracking records, for backup purposes. If you do not need this, choose &amp;quot;Delete remaining records&amp;quot; which will remove them.&amp;nbsp;&lt;/p&gt;
&lt;p&gt;If you simply want to remove all change tracking records, set the &amp;ldquo;records to keep&amp;rdquo;&amp;nbsp;to zero and choose &amp;quot;Delete remaining&amp;quot;. Perform a database compact (File &amp;gt; Database Utilities &amp;gt; Compact) after doing this, to free up the space. Consider turning off Change Tracking (from the Review menu) if you do not wish to continue accumulating more change tracking records.&lt;/p&gt;
&lt;h2&gt;Solution - Compact Database&lt;/h2&gt;
&lt;p&gt;Another way to reduce the size of the database is to try&amp;nbsp;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option. (Note: if you utilize the GIS-ID feature, you may need to purge additional elements from the &amp;quot;GIS-ID Center for Deleted Elements&amp;quot; - see &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting/"&gt;this article&lt;/a&gt; for more)&lt;br /&gt;&lt;br /&gt;&lt;/p&gt;
&lt;h2 id="mcetoc_1gorgaoka2"&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h2&gt;
&lt;p&gt;If you are using an older version of the software, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1 id="mcetoc_1gorgaoka3"&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, Reviewed2023, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/12</link><pubDate>Mon, 01 May 2023 21:41:08 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Larry Abla</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 12 posted to OpenFlows | Water Infrastructure Wiki by Larry Abla on 5/1/2023 9:41:08 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1 id="mcetoc_1gorgaoka0"&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1 id="mcetoc_1gorgaoka1"&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, check if you have &lt;strong&gt;Change Tracking&lt;/strong&gt; enabled. This is located under the Review menu. Click Change Tracking and see if the box is checked next to &amp;quot;Track Changes&amp;quot;. If so, choose &amp;quot;view tracked changes&amp;quot; and check the number of records. If it is large (for example millions of rows), this can cause the database file to be very large and reduce performance especially when opening and saving the model. This could be caused by performing bulk operations such as ModelBuilder imports and constraint based design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&amp;nbsp;&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08 (and greater), you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent.&lt;/p&gt;
&lt;p&gt;In addition, a Bulk Archive features was added in the CONNECT Edition Update 4 release (10.04.00.XXX). With this, you will be able to archive a large number of items from the Change Tracking database. This will remove the items from the model database file and store the archived changes in an external database file.&lt;/p&gt;
&lt;p&gt;You can access this by going to the Review tab&lt;span&gt;, click &amp;quot;Bulk Archive.&amp;quot; You can choose how many records to keep or delete all change tracking data. After performing the Bulk Archine, it is recommended to&amp;nbsp;&lt;/span&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compact the database&lt;/a&gt;&lt;span&gt;.&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;&lt;strong&gt;Note:&lt;/strong&gt;&amp;nbsp;This does not replace the Archive feature in the Change Tracking manager.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;For more information, see the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/2806.BulkArchive1.png" /&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/1348.BulkArchive2.png" /&gt;&lt;/p&gt;
&lt;p&gt;Archive remaining records means it will save a .sqlite file, which is unlikely to be useful.&amp;nbsp;Delete remaining records means it will remove them.&amp;nbsp;&lt;/p&gt;
&lt;p&gt;If you&amp;nbsp;want to get rid of everything, set the &amp;ldquo;records to keep&amp;rdquo;&amp;nbsp;to zero and choose Delete remaining.&lt;/p&gt;
&lt;p&gt;&lt;span&gt;This of course assumes you don&amp;rsquo;t want to keep any of the database records (you might want to keep for example the past few day&amp;#39;s records). &lt;/span&gt;&lt;/p&gt;
&lt;p&gt;Another way to reduce the size of the database is to try&amp;nbsp;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option. (Note: if you utilize the GIS-ID feature, you may need to purge additional elements from the &amp;quot;GIS-ID Center for Deleted Elements&amp;quot; - see &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting/"&gt;this article&lt;/a&gt; for more)&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h3 id="mcetoc_1gorgaoka2"&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1 id="mcetoc_1gorgaoka3"&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, Reviewed2023, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/11</link><pubDate>Wed, 08 Mar 2023 20:12:03 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Scott Kampa</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 11 posted to OpenFlows | Water Infrastructure Wiki by Scott Kampa on 3/8/2023 8:12:03 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1 id="mcetoc_1gorgaoka0"&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1 id="mcetoc_1gorgaoka1"&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, check if you have &lt;strong&gt;Change Tracking&lt;/strong&gt; enabled. This is located under the Review menu. Click Change Tracking and see if the box is checked next to &amp;quot;Track Changes&amp;quot;. If so, choose &amp;quot;view tracked changes&amp;quot; and check the number of records. If it is large (for example millions of rows), this can cause the database file to be very large and reduce performance especially when opening and saving the model. This could be caused by performing bulk operations such as ModelBuilder imports and constraint based design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&amp;nbsp;&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08 (and greater), you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent.&lt;/p&gt;
&lt;p&gt;In addition, a Bulk Archive features was added in the CONNECT Edition Update 4 release (10.04.00.XXX). With this, you will be able to archive a large number of items from the Change Tracking database. This will remove the items from the model database file and store the archived changes in an external database file.&lt;/p&gt;
&lt;p&gt;You can access this by going to the Review tab&lt;span&gt;, click &amp;quot;Bulk Archive.&amp;quot; You can choose how many records to keep or delete all change tracking data. After performing the Bulk Archine, it is recommended to&amp;nbsp;&lt;/span&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compact the database&lt;/a&gt;&lt;span&gt;.&amp;nbsp;&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;&lt;span&gt;&lt;strong&gt;Note:&lt;/strong&gt;&amp;nbsp;This does not replace the Archive feature in the Change Tracking manager.&lt;/span&gt;&lt;/p&gt;
&lt;p&gt;For more information, see the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/2806.BulkArchive1.png" /&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/1348.BulkArchive2.png" /&gt;&lt;/p&gt;
&lt;p&gt;Another way to reduce the size of the database is to try&amp;nbsp;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option. (Note: if you utilize the GIS-ID feature, you may need to purge additional elements from the &amp;quot;GIS-ID Center for Deleted Elements&amp;quot; - see &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting/"&gt;this article&lt;/a&gt; for more)&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h3 id="mcetoc_1gorgaoka2"&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1 id="mcetoc_1gorgaoka3"&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, Reviewed2023, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/10</link><pubDate>Thu, 09 Feb 2023 16:43:07 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Jesse Dringoli</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 10 posted to OpenFlows | Water Infrastructure Wiki by Jesse Dringoli on 2/9/2023 4:43:07 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1 id="mcetoc_1gorgaoka0"&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1 id="mcetoc_1gorgaoka1"&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, check if you have &lt;strong&gt;Change Tracking&lt;/strong&gt; enabled. This is located under the Review menu. Click Change Tracking and see if the box is checked next to &amp;quot;Track Changes&amp;quot;. If so, choose &amp;quot;view tracked changes&amp;quot; and check the number of records. If it is large (for example millions of rows), this can cause the database file to be very large and reduce performance especially when opening and saving the model. This could be caused by performing bulk operations such as ModelBuilder imports and constraint based design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&amp;nbsp;&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08 (and greater), you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. From the Review menu, click &amp;quot;Bulk Archive&amp;quot;, choose how many records to keep (for example if you only want to remove a certain amount of old records), and click OK. Then, &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compact the database&lt;/a&gt;. For more information, see the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/2806.BulkArchive1.png" /&gt;&lt;/p&gt;
&lt;p&gt;&lt;img alt=" " border="0" src="/cfs-file/__key/communityserver-wikis-components-files/00-00-00-00-32/1348.BulkArchive2.png" /&gt;&lt;/p&gt;
&lt;p&gt;Another way to reduce the size of the database is to try&amp;nbsp;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option. (Note: if you utilize the GIS-ID feature, you may need to purge additional elements from the &amp;quot;GIS-ID Center for Deleted Elements&amp;quot; - see &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting/"&gt;this article&lt;/a&gt; for more)&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h3 id="mcetoc_1gorgaoka2"&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1 id="mcetoc_1gorgaoka3"&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/9</link><pubDate>Thu, 09 Feb 2023 16:28:59 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Jesse Dringoli</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 9 posted to OpenFlows | Water Infrastructure Wiki by Jesse Dringoli on 2/9/2023 4:28:59 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, check if you have &lt;strong&gt;Change Tracking&lt;/strong&gt; enabled. This is located under the Review menu. Click Change Tracking and see if the box is checked next to &amp;quot;Track Changes&amp;quot;. If so, choose &amp;quot;view tracked changes&amp;quot; and check the number of records. If it is large (for example millions of rows), this can cause the database file to be very large and reduce performance especially when opening and saving the model. This could be caused by performing bulk operations such as ModelBuilder imports and constraint based design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&amp;nbsp;&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08 (and greater), you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. See the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Another way to reduce the size of the database is to try&amp;nbsp;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option. (Note: if you utilize the GIS-ID feature, you may need to purge additional elements from the &amp;quot;GIS-ID Center for Deleted Elements&amp;quot; - see &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting/"&gt;this article&lt;/a&gt; for more)&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/8</link><pubDate>Fri, 01 Jul 2022 16:29:53 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Jesse Dringoli</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 8 posted to OpenFlows | Water Infrastructure Wiki by Jesse Dringoli on 7/1/2022 4:29:53 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option. (Note: if you utilize the GIS-ID feature, you may need to purge additional elements from the &amp;quot;GIS-ID Center for Deleted Elements&amp;quot; - see &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting/"&gt;this article&lt;/a&gt; for more)&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the &lt;strong&gt;Change Tracking&lt;/strong&gt; feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and constraint based design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08, you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. See the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/7</link><pubDate>Mon, 25 Apr 2022 08:29:57 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Yashodhan Joshi</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 7 posted to OpenFlows | Water Infrastructure Wiki by Yashodhan Joshi on 4/25/2022 8:29:57 AM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option.&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the &lt;strong&gt;Change Tracking&lt;/strong&gt; feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and constraint based design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08, you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. See the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/6</link><pubDate>Tue, 19 Apr 2022 14:00:23 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Sushma Choure</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 6 posted to OpenFlows | Water Infrastructure Wiki by Sushma Choure on 4/19/2022 2:00:23 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option.&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the &lt;strong&gt;Change Tracking&lt;/strong&gt; feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and automated design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08, you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. See the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/pastedimage1650377098119v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/5</link><pubDate>Tue, 19 Apr 2022 13:58:28 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Sushma Choure</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 5 posted to OpenFlows | Water Infrastructure Wiki by Sushma Choure on 4/19/2022 1:58:28 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option.&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the &lt;strong&gt;Change Tracking&lt;/strong&gt; feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and automated design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08, you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. See the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load. Make sure you are looking at the correct database, SQLITE expert saves earlier opened databases as well.&amp;nbsp;&lt;/li&gt;
&lt;li&gt;&lt;img alt=" " src="/resized-image/__size/450x450/__key/communityserver-wikis-components-files/00-00-00-00-32/4405.pastedimage1650376834244v1.png" /&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table. If you see the file is read-only and not able to make changes, run&amp;nbsp;&lt;span&gt;SQLITE Expert as an admin. (right click on&amp;nbsp;SQLITE Expert from start menu &amp;gt; run as admin).&amp;nbsp;&lt;/span&gt;&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/4</link><pubDate>Thu, 05 Nov 2020 13:35:11 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Jesse Dringoli</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 4 posted to OpenFlows | Water Infrastructure Wiki by Jesse Dringoli on 11/5/2020 1:35:11 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option.&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the &lt;strong&gt;Change Tracking&lt;/strong&gt; feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and automated design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows).&lt;/p&gt;
&lt;p&gt;Starting with&amp;nbsp;version 10.03.01.08, you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. See the section &amp;quot;Removing / clearing change tracking records&amp;quot; in the following article: &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking for insights into model changes&lt;/a&gt;&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup for older versions (advanced)&lt;/h3&gt;
&lt;p&gt;If you are using an older version, you can use the below information to manually clean up the change tracking database using the below information, or contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table.&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/3</link><pubDate>Mon, 17 Aug 2020 04:56:36 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Yashodhan Joshi</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 3 posted to OpenFlows | Water Infrastructure Wiki by Yashodhan Joshi on 8/17/2020 4:56:36 AM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;p&gt;&lt;span style="color:#ff0000;"&gt;&lt;strong&gt;Note:&amp;nbsp;&lt;/strong&gt;&lt;/span&gt;&lt;em&gt;Starting with version 10.03.01.08, you can now archive change tracking data in both the water and storm-sewer products.&amp;nbsp;This can help with situations where including change tracking data may otherwise increase the size of the model&amp;#39;s database file to a great extent. If however, you still wish to clean the change tracking data or are working on a lower version, see the below solution.&lt;/em&gt;&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option.&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the &lt;strong&gt;Change Tracking&lt;/strong&gt; feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and automated design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows). A future version may include enhancements to avoid this problem or prompt (reference PBI # 136596), but in the meantime, if you find that the Change Tracking feature (under the Review menu) contains a very large amount of data, please contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database, or use the below advanced change tracking cleanup steps.&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup (advanced)&lt;/h3&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table.&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/2</link><pubDate>Thu, 13 Jun 2019 21:05:40 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Jesse Dringoli</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 2 posted to OpenFlows | Water Infrastructure Wiki by Jesse Dringoli on 6/13/2019 9:05:40 PM&lt;br /&gt;
&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h1&gt;&lt;br /&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option.&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the &lt;strong&gt;Change Tracking&lt;/strong&gt; feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and automated design runs. These types of operations can result in a large number of extra rows of data in the change tracking system (sometimes tens of millions of rows). A future version may include enhancements to avoid this problem or prompt (reference PBI # 136596), but in the meantime, if you find that the Change Tracking feature (under the Review menu) contains a very large amount of data, please contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database, or use the below advanced change tracking cleanup steps.&lt;/p&gt;
&lt;h3&gt;Change Tracking Cleanup (advanced)&lt;/h3&gt;
&lt;p&gt;Note: the following information is for advanced users who are comfortable with directly editing the model database. &lt;span style="color:#ff0000;"&gt;Care should be taken to back up your model before performing these steps.&lt;/span&gt; If in doubt, contact Technical Support.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;strong&gt;Download and install a SQLITE database editor&lt;/strong&gt;. These instructions use &lt;a href="http://www.sqliteexpert.com/download.html"&gt;SQLITE Expert Personal&lt;/a&gt; (version 5.3.4) as an example. Ensure that you use a 64-bit version, as this will be needed for large database sizes.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Back up your model database &lt;/strong&gt;- make a copy&amp;nbsp;of the model&amp;#39;s SQLITE file.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Open a copy&lt;/strong&gt; of the original SQLITE file (File &amp;gt; Open Database, in SQLITE Expert). Opening a copy of the original will ensure there is a clean backup in case anything goes wrong.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Check the number of rows&lt;/strong&gt; in the Change Tracking Table
&lt;ol&gt;
&lt;li&gt;Ensure that table counts are displayed. In SQLITE Expert, go to Tools &amp;gt; Options &amp;gt; General and make sure the box is checked for &amp;quot;show record counts&amp;quot;&lt;/li&gt;
&lt;li&gt;Scroll down the list of tables on the left side until you see the table named&amp;nbsp;HMIChangeTracking. The number of records should be shown in parenthesis next to it. This make take some time to load.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Clear the Change Tracking Table&lt;/strong&gt;
&lt;ol&gt;
&lt;li&gt;Select the HMIChangeTracking table in the list on the left&lt;/li&gt;
&lt;li&gt;Select the SQL tab on the right (in SQLITE Expert)&lt;/li&gt;
&lt;li&gt;In the SQL tab, use the following SQL statement to delete all rows from the HMIChangeTracking table.&lt;br /&gt;&lt;br /&gt;&lt;em&gt;&lt;strong&gt;DELETE FROM HMIChangeTracking&lt;/strong&gt;&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;For users with SQL knowledge: you can add a WHERE statement following HMIChangeTracking to limit which rows to actually delete.&lt;/li&gt;
&lt;li&gt;Click the Execute SQL button.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;li&gt;Once the execution completes, go to Database -&amp;gt; Vacuum. This will essentially do a compact database and vacuum up all the now empty space after such a large deletion.&amp;nbsp; The vacuum may take a few moments to complete.&lt;/li&gt;
&lt;li&gt;Check the row count again - it should be zero if you used the above query to delete all rows.&lt;/li&gt;
&lt;li&gt;Close out of the SQLITE editor application, check the SQLITE file size, open the model and ensure that it is working properly.&lt;/li&gt;
&lt;/ol&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item><item><title>SQLITE database size grows very large due to Change Tracking</title><link>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking/revision/1</link><pubDate>Mon, 10 Jun 2019 15:53:03 GMT</pubDate><guid isPermaLink="false">6dad98f5-dbc9-4c4d-a9ba-e9da8dc6aa8e:97acb004-19d6-464d-986b-b1637b6ee791</guid><dc:creator>Jesse Dringoli</dc:creator><comments>https://communities.bentley.com/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/42552/sqlite-database-size-grows-very-large-due-to-change-tracking#comments</comments><description>Revision 1 posted to OpenFlows | Water Infrastructure Wiki by Jesse Dringoli on 6/10/2019 3:53:03 PM&lt;br /&gt;

&lt;table style="background-color:#dce5f0;border:0px solid #dce5f0;" border="0"&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Product(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;WaterCAD, WaterGEMS, HAMMER, SewerGEMS, SewerCAD, StormCAD, CivilStorm&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Version(s):&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;10.02.XX.XX and greater&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style="text-align:right;"&gt;&lt;strong&gt;Area:&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Output and Reporting&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h1&gt;Problem&lt;/h1&gt;
&lt;p&gt;The .SQLITE model database file is extremely large, even for a model with a reasonable number of elements and alternatives. For example 6 GB for a 7000 pipe model. This may make performance very sluggish and take a long time to open and save.&lt;/p&gt;
&lt;p&gt;&lt;/p&gt;
&lt;h1&gt;Solution&lt;/h1&gt;
&lt;p&gt;First, try &lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;compacting the database &lt;/a&gt;and purging records to save space, in case old records of previously removed elements and components are still stored in the database. This is done by opening the product without opening any models, then going to File &amp;gt; Database Utilities &amp;gt; Compact Database &amp;gt; Select Model &amp;gt; choose the Purge option.&lt;/p&gt;
&lt;p&gt;If compacting the database does not help, This issue has been known to occur when the Change Tracking feature has been turned on and the user has performed bulk operations such as ModelBuilder imports and automated design runs. These types of operations can result in many extra rows of data in the change tracking system. A future version may include enhancements to avoid this problem or prompt (reference PBI # 136596), but in the meantime, if you find that the Change Tracking feature (under the Review menu) contains a very large amount of data, please contact Technical Support (for example post in the &lt;a href="/products/hydraulics___hydrology/f/haestad-hydraulics-and-hydrology-forum"&gt;Forum&lt;/a&gt;) to manually remove the information from the database.&lt;/p&gt;
&lt;h1&gt;See Also&lt;/h1&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/37946/using-change-tracking-for-insights-into-model-changes"&gt;Using Change Tracking&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="/products/hydraulics___hydrology/w/hydraulics_and_hydrology__wiki/9032/compacting-a-database-for-optimal-performance-and-general-troubleshooting"&gt;Compacting a database for optimal performance and general troubleshooting&lt;/a&gt;&lt;/p&gt;&lt;div style="clear:both;"&gt;&lt;/div&gt;

&lt;div style="font-size: 90%;"&gt;Tags: HAMMER, StormCAD, SewerGEMS, SewerCAD, WaterGEMS, Change Tracking, CivilStorm, WaterCAD, SQLite&lt;/div&gt;
</description></item></channel></rss>