AgE 2.7 : Monitoring

Main idea of Monitoring

Monitoring uses RxJava terminology and consists of the following elements:

• Supplier - gets the data.
• Observable - calls Supplier to provide the data in a given time interval, wraps received data to and make them 'observable'. Observable can push data to many Handlers.
• Handler - in the simplest case (BasicHandlerFactory) only merges the data from many Observables into the single stream; in others cases Handler can process the data and e.g. calculate the avarage from the given data stream. Handler can receive the data from many Observables, and can push the data to many Observers.
• Observer - receives the data from forwarded by Handler and make them useful for user. Observer can receive the data from many Handlers.
• Others utility components.

Maven dependencies

To use monitoring, the following dependencies are required.

<dependency>
	<groupId>org.jage.monitoring</groupId>
	<artifactId>monitoring-core</artifactId>
	<version>${project.version}</version>
</dependency>

<dependency>
	<groupId>org.jage.monitoring</groupId>
	<artifactId>age-visualization</artifactId>
	<version>${project.version}</version>
</dependency>

<dependency>
	<groupId>org.jage.monitoring</groupId>
	<artifactId>age-bindings</artifactId>
	<version>${project.version}</version>
</dependency>

Monitoring configuration

Configuration of monitoring can be done in the following ways:

• in TypeSafe file - recomended way,
• in XML node configuration file - NOT recomended because of its complexity,
• mixed way when the main components are defined in TypeSafe file but some Suppliers are defined in XML node configuration file,

This is the mandatory content of XML node configuration file which have to occur, regardless which way of configuration you will choose.

<component class="org.jage.lifecycle.AutoExitHook" />
<block name="monitoring">
	<component class="org.jage.monitoring.Monitoring">
		<component class="org.jage.monitoring.config.DefaultComputationInstanceProvider"/>
		<component class="org.jage.monitoring.config.ExecutorProvider"/>
		<component class="org.jage.monitoring.config.ExecutorShutdownCaller"/>
		<component class="org.jage.monitoring.config.RxComputationSchedulerProvider"/>
		...
	</component>
</block>

Configuration via TypeSafe Config

To enable the possibility of configuration via TypeSafe Config, in monitoring block (see above) as an argument of org.jage.monitoring.Monitoring component's constructor, the TypeSafeConfig component must be declared. Its constructor accepts the path to the file with monitoring configuration via TypeSafe. The filename should contain extension conf. However, string describing path and filename in TypeSafeConfig should not contain conf extension.
Example. The file monitoring.conf is located in folder examples/monitoring

<component class="org.jage.monitoring.config.TypeSafeConfig">
	<constructor-arg value="examples/monitoring/monitoring" />
</component>

TypeSafe Config reduces the necessity of using quotation marks. However, in case where the value of some argument contains some chars like @ : , / the quoutation marks are needed.

The example of monitoring TypeSafe configuration file

age.monitoring {
	observables {
		randomObservable {
			type = class
			arg = org.jage.monitoring.supplier.RandomSupplier
			rate = 1000 ms
		}
		queryObservable {
			type = query
			arg = "/TheXFiles/Mulder@alienAbductions"
			rate = 1500 ms
		}
	}
	
	observers {
		console { type = console}
		
		live {
			type = visualization
			url = "http://localhost:8080/rest/emas"
		}
		
		db1 {
			type = database
			url = "jdbc:postgresql://localhost:5432/postgres/mojemas"
			user = postgres
			password = 12345678
		}
		
		db2 {
			type = nosqldatabase
			url = "mongo://localhost/age/emas"
		}
	}
	
	handlers {
		randomness {
			observable = [randomObservable]
			observer = [console, db1]
			class = org.jage.monitoring.handler.BasicHandlerFactory
			method = "create"	
		}

		avgOfTwo {
			observable = [randomObservable , queryObservable ]
			observer = [console]
			class = org.jage.monitoring.handler.AvgHandlerFactory
			method = "create"	
		}
}

Below you can see the content of XML node configuration file, which is needed to enable TypeSafe monitoring configuration.

<component class="org.jage.lifecycle.AutoExitHook" />
<block name="monitoring">
	<component class="org.jage.monitoring.Monitoring">
		<component class="org.jage.monitoring.config.DefaultComputationInstanceProvider"/>
		<component class="org.jage.monitoring.config.ExecutorProvider"/>
		<component class="org.jage.monitoring.config.ExecutorShutdownCaller"/>
		<component class="org.jage.monitoring.config.RxComputationSchedulerProvider"/>
		<constructor-arg>
			<component class="org.jage.monitoring.config.TypeSafeConfig">
				<constructor-arg value="examples/monitoring/monitoring" />
			</component>
		</constructor-arg>
	</component>
</block>

To see more details visit Suppliers in TypeSafe and Suppliers/Observables in TypeSafe.

Configuration via XML

The correct XML monitoring configuration should be included in block named monitoring as an argument of org.jage.monitoring.Monitoring component's constructor, XmlConfig must contain definitions the following elements:

• component which implements interface ComputationInstanceProvider; it provides the way of creating the computationInstance parameters, which is required to run most of the observers components; default implementation is defined in the DefaultComputationInstanceProvider class,
• components ExecutorProvider, ExecutorShutdownCaller, RxComputationSchedulerProvider,
• at least one supplier component, which implements interface Supplier; see more Suppliers,
  • to use the supplier, ObservableProvider have to be created. It binds supplier with rate and in this way observable is created. Constructor of ObservableProvider accepts the reference to supplier and rate,
• at least one observer component, which implements interface Observer; see more Observers,
• at least one component HandlerFactoryProvider which constructor accepts the following elements:
  • name - handler's name, 
  • observableProviders - list with reference(s) to defined observable provider(s),
  • observers - list with reference(s) to defined observer(s),
  • clazz - qualified class name of handler which will be used,
  • method - name of method which will be executed; In this method observable(s) will be subscribed on observer(s).

Complete example

<component class="org.jage.lifecycle.AutoExitHook" />
	
<block name="monitoring">
	<component class="org.jage.monitoring.Monitoring">
			<component class="org.jage.monitoring.config.DefaultComputationInstanceProvider"/>
			<component class="org.jage.monitoring.config.ExecutorProvider"/>
			<component class="org.jage.monitoring.config.ExecutorShutdownCaller"/>
			<component class="org.jage.monitoring.config.RxComputationSchedulerProvider"/>
			
			<component name="stringQuery" class="org.jage.monitoring.supplier.AgentStringQuerySupplier">
				<constructor-arg name="query" value="/TheXFiles/Scully@alienAbductions"/>
			</component>
	
			<component name="random" class="org.jage.monitoring.supplier.RandomSupplier" />
	
			<component name="randomProvider" class="org.jage.monitoring.observable.ObservableProvider">
				<constructor-arg name="supplier" ref="random"/>
				<constructor-arg name="rate" value="1000" type="Long"/>
			</component>
	
			<component name="queryProvider" class="org.jage.monitoring.observable.ObservableProvider">
				<constructor-arg name="supplier" ref="stringQuery"/>
				<constructor-arg name="rate" value="1000" type="Long"/>
			</component>
	
			<list name="observers">
				<component name="logging" class="org.jage.monitoring.observer.LoggingObserver" />
			</list>
			<constructor-arg>
				<component class="org.jage.monitoring.config.XmlConfig">
					<constructor-arg>
						<list>
							<component name="handler1" class="org.jage.monitoring.handler.HandlerFactoryProvider">
								<constructor-arg name="name" value="handler1" />
								<constructor-arg name="observableProviders">
									<list name="observableProviders">
										<reference target="queryProvider" />
										<reference target="randomProvider" />
									</list>
								</constructor-arg>
								<constructor-arg name="observers" ref="observers" />
								<constructor-arg name="clazz" value="org.jage.monitoring.handler.AvgHandlerFactory" />
								<constructor-arg name="method" value="create" />
							</component>
			
							<component name="handler2" class="org.jage.monitoring.handler.HandlerFactoryProvider">
								<constructor-arg name="name" value="handler2" />
								<constructor-arg name="observableProviders">
									<list>
										<reference target="queryProvider" />
									</list>
								</constructor-arg>
								<constructor-arg name="observers" ref="observers" />
								<constructor-arg name="clazz" value="org.jage.monitoring.handler.BasicHandlerFactory" />
								<constructor-arg name="method" value="create" />
							</component>
						</list>
					</constructor-arg>
				</component>
			</constructor-arg>
		</component>
	</block>

Suppliers

Available Suppliers:

• custom class created by user which implements com.google.common.base.Supplier interface; example org.jage.monitoring.supplier.RandomSupplier,
• org.jage.monitoring.supplier.AgentQuerySupplier which constructor accepts the class which implements org.jage.query.IQuery interface. AgenQuerySupplier basing on provided class, performs appropriate query,
• org.jage.monitoring.supplier.AgentStringQuerySupplier which constructor accepts query in string format. The general form of the query: pathToAgent@property1,propertyN; example /Mulder@alienAbductions.

Suppliers/Observables in TypeSafe

In TypeSafe configuration file Suppliers are not defined in a direct way. They are part of the Observable
All observables defined in TypeSafe configuration file and used in handlers, must be placed in observables tag. The general form of defininig observable:

[observable_name] {
   type = (query | class | ref)
   arg = [value]
   rate = [value][unit]
}

Observers

Available observers:

• org.jage.monitoring.observer.LoggingObserver - displays the data in the following format: handler_name (timestamp) data; takes no parameters,
• org.jage.monitoring.observer.DatabaseObserver - save the data in pointed PostgreSQL database; takes the following parameters:
  • url - in form jdbc:postgresql://hostname:5432/schemaname/computationtype (i.e. jdbc:postgresql://localhost:5432/postgres/myemas)
  • user
  • password
• org.jage.monitoring.observer.NoSqlDatabaseObserver - save the data in pointed MongoDB NoSQL database; takes the parameter:
  • url - in form mongo://hostname/schemaname/computationtype (i.e. mongo://localhost/age/emas)
• org.jage.monitoring.observer.VisualizationObserver - passes the data to webapplication which can create a live chart; takes the parameter:

  • url - http://hostname:8080/rest/computationtype (i.e. http://localhost:8080/rest/myemas).

     

Most of observers (all except console) bind two addition parameters with each handler. Each data which is pushed to bserver can be identified by three parameters named: computation type, computation instance and handler name. 

For each handler, the NoSqlDatabaseObserver class observer creates a new collection (equivalent of table in relational model), which name look like this
computationtype_computationinstance_handlername, i.e.  myemas_1377243932882_fitness.

All created observers which use computation instance assume that the value of this parameter indicates the timestamp generated in the moment when the computation was started. Component which provides that value must implements org.jage.monitoring.config.ComputationInstanceProvider.

Observers in TypeSafe

All observers defined in TypeSafe configuration file and used in handlers, must be placed in observers tag.

The general form of defininig observer:

[observer_name] {
   type = (console | database | nosqldatabase | visualization)
   [parameter] = [value]
}

All the parameters of observers in XML configuration way, are the same in TypeSafe Config.

Time is absolute

Time which is gathered in parallel with monitoring data is the absolute time. So if you need to relative time, create your own handler which will convert time from absolute to relative or perform such transformation beyond monitoring.