The so called baseline of layers is saved in an Extension Dictionary on each layer and adds of course to the size of the file.
I made a test drawing with 1000 layers. Not so much if you work with many xrefs that has many layers. With all layers reconciled the size was 113 KB. After purging with the code below the size come down to 81 KB. 32 extra KB per 1000 layers. No big deal but here is a way to purge them using AutoLISP.
Download for PurgeReconciledLayers.LSP
;;; PurgeReconciledLayers.LSP ;;; ;;; By Jimmy Bergmark ;;; Copyright (C) 2007 JTB World, All Rights Reserved ;;; Website: www.jtbworld.com ;;; E-mail: email@example.com ;;; 2007-04-05 - First release ;;; Written for AutoCAD 2008 ;;; Purge all information about reconciled layers in the drawing (defun PurgeReconciledLayers () (vl-load-com) (vlax-for layer (vla-get-Layers (vla-get-ActiveDocument (vlax-get-acad-object) ) ) (vl-Catch-All-Apply '(lambda () (vla-Remove (vla-GetExtensionDictionary layer ) "ADSK_XREC_LAYER_RECONCILED" ) ) ) (vl-Catch-All-Apply '(lambda () (vla-delete (vla-GetExtensionDictionary layer ) ) ) ) ) (setvar "LAYEREVAL" 0) (setvar "LAYERNOTIFY" 0) (princ) ) ; Remove the row below if you don't want to run the code automatically when the AutoLISP file is loaded. (PurgeReconciledLayers)