Caucho maker of Resin Server | Application Server (Java EE Certified) and Web Server


 

Resin Documentation

home company blog wiki docs 
app server web server 
health cloud java ee pro 
 Resin Server | Application Server (Java EE Certified) and Web Server
 

health meters


Health meters are a simple way to create visually pleasing graphs in /resin-admin.

Configuration

health.xml

Meters are configured as part of health.xml using CanDI to create and update Java objects. Refer to health checking configuration for a full description of health.xml. Resin 4.0.17 and later includes a full compliemnt of pre-configured JMX meters in health.xml.

Example: importing health.xml into resin.xml
<resin xmlns="http://caucho.com/ns/resin"
       xmlns:resin="urn:java:com.caucho.resin">
  <cluster-default>  
    ...
    <!--
       - Admin services
      -->
    <resin:DeployService/>
    
    <resin:if test="${resin.professional}">
      <resin:AdminServices/>
    </resin:if>

    <!--
       - Configuration for the health monitoring system
      -->
    <resin:if test="${resin.professional}">
      <resin:import path="${__DIR__}/health.xml" optional="true"/>
    </resin:if>
    ...
  </cluster-default>
</resin>

Note: <resin:AdminServices/> (or more precisely just <resin:StatsService/>) is required to support health meters and graphing.

Meter names

Health meters are named using a concatenation of keys separated by pipe (|) characters, loosely organized from least specific to most specific. Since meter statistics are shared between each member in a Resin cluster, Resin will automatically prefix each meter name with the cluster node index to insure the name is unique between cluster members.

The pipe character in the name provides a secondary benefit of helping to enhance the /resin-admin UI by categorizing meters into drill downs. Consider the following example.

Example: meter naming
<cluster xmlns="http://caucho.com/ns/resin"
         xmlns:resin="urn:java:com.caucho.resin"
         xmlns:health="urn:java:com.caucho.health"
         xmlns:ee="urn:java:ee">

  <health:JmxDeltaMeter>
    <name>JVM|Compilation|Compilation Time</name>
    <object-name>java.lang:type=Compilation</object-name>
    <attribute>TotalCompilationTime</attribute>
  </health:JmxDeltaMeter>

</cluster>

In this example, JVM|Compilation|Compilation Time provides the base of the name. For cluster node index 0, Resin prefixes the name with 00|. /resin-admin will then use the cluster index and first two keys to create drill downs to logically organized meters for display.

Graphs: 00|JVM|Compilation|Compilation Time, Time:1 Hour

JMX meters

Virtually any local numeric JMX MBean attribute can be graphed using JMX meters.

<health:JmxMeter>

child of cluster

Creates a meter that graphs the current value of a numeric JMX MBean attribute.

<health:JmxMeter> Attributes
ATTRIBUTEDESCRIPTIONTYPEDEFAULT
nameThe name of the meter to display in /resin-admin (see meter names)StringN/A
objectNameThe JMX MBean nameStringN/A
attributeThe MBean attribute to sampleStringN/A
Example: <health:JmxMeter> in health.xml
<cluster xmlns="http://caucho.com/ns/resin"
         xmlns:resin="urn:java:com.caucho.resin"
         xmlns:health="urn:java:com.caucho.health"
         xmlns:ee="urn:java:ee">

  <health:JmxMeter>
    <name>OS|Memory|Physical Memory Free</name>
    <object-name>java.lang:type=OperatingSystem</object-name>
    <attribute>FreePhysicalMemorySize</attribute>
  </health:JmxMeter>

</cluster>

<health:JmxDeltaMeter>

child of cluster

Creates a meter that graphs the difference between the current and previous values of a numeric JMX MBean attribute.

<health:JmxDeltaMeter> Attributes
ATTRIBUTEDESCRIPTIONTYPEDEFAULT
nameThe name of the meter to display in /resin-admin (see meter names)StringN/A
objectNameThe JMX MBean nameStringN/A
attributeThe MBean attribute to sampleStringN/A
Example: <health:JmxDeltaMeter> in health.xml
<cluster xmlns="http://caucho.com/ns/resin"
         xmlns:resin="urn:java:com.caucho.resin"
         xmlns:health="urn:java:com.caucho.health"
         xmlns:ee="urn:java:ee">

  <health:JmxDeltaMeter>
    <name>JVM|Compilation|Compilation Time</name>
    <object-name>java.lang:type=Compilation</object-name>
    <attribute>TotalCompilationTime</attribute>
  </health:JmxDeltaMeter>

</cluster>

Statistical Analysis

Detecting Anomalies

Meters alone are useful for manual inspection in resin-admin since every meter can be graphed. However Resin provides an extremely useful automatic analysis tool called AnomalyAnalyzer. AnomalyAnalyzer looks at the current meter value, checking for deviations from the average value. So unusual changes like a spike in blocked threads can be detected.

<health:AnomalyAnalyzer>

child of cluster

AnomalyAnalyzer examines a meter value, checking for deviations from the average value. So unusual changes like a spike in blocked threads can be detected, logged, and trigger health actions.

<health:AnomalyAnalyzer> Attributes
ATTRIBUTEDESCRIPTIONTYPEDEFAULT
meterName of the meter to analyze (ie. from <health:JmxMeter>)Stringrequired
health-eventA string to use to match using <health:IfHealthEvent>StringNone: when absent no health event will fire
min-samplesMinimum number of samples required to calculate an averageint60 (typically 1 hour of data)
sigma-thresholdThe number of standard deviations for a sample to be considered an anomaly.int5
Example: <health:AnomalyAnalyzer> in health.xml
<cluster xmlns="http://caucho.com/ns/resin"
         xmlns:resin="urn:java:com.caucho.resin"
         xmlns:health="urn:java:com.caucho.health"
         xmlns:ee="urn:java:ee">
         
  <health:JmxMeter>
    <name>JVM|Thread|JVM Blocked Count</name>
    <objectName>resin:type=JvmThreads</objectName>
    <attribute>BlockedCount</attribute>
  </health:JmxMeter>

  <health:AnomalyAnalyzer>
    <meter>JVM|Thread|JVM Blocked Count</meter>
    <health-event>caucho.thread.anomaly.jvm-blocked</health-event>
  </health:AnomalyAnalyzer>

  <health:DumpThreads>
    <health:IfHealthEvent regexp="caucho.thread"/>
    <health:IfNotRecent time="15m"/>
  </health:DumpThreads>
  
</cluster>

Reacting to Anomalies

The <health-event> attribute of AnomalyAnalyzer allows us to tie health actions to a detected anomaly by using the <health:IfHealthEvent> condition.

<health:IfHealthEvent>

child of cluster

Causes an action to fire in response to a matching health event. This is usually used in combination with AnomalyAnalyzer with a <health-event> attribute.

<health:IfHealthEvent> Attributes
ATTRIBUTEDESCRIPTIONTYPEDEFAULT
regexpA regular expression the event must match.java.util.regex.Patternrequired
Example: <health:IfHealthEvent> in health.xml
<cluster xmlns="http://caucho.com/ns/resin"
         xmlns:resin="urn:java:com.caucho.resin"
         xmlns:health="urn:java:com.caucho.health"
         xmlns:ee="urn:java:ee">
         
  <health:JmxMeter>
    <name>JVM|Thread|JVM Blocked Count</name>
    <objectName>resin:type=JvmThreads</objectName>
    <attribute>BlockedCount</attribute>
  </health:JmxMeter>

  <health:AnomalyAnalyzer>
    <meter>JVM|Thread|JVM Blocked Count</meter>
    <health-event>caucho.thread.anomaly.jvm-blocked</health-event>
  </health:AnomalyAnalyzer>

  <health:DumpThreads>
    <health:IfHealthEvent regexp="caucho.thread"/>
    <health:IfNotRecent time="15m"/>
  </health:DumpThreads>
  
</cluster>

Copyright © 1998-2012 Caucho Technology, Inc. All rights reserved. Resin ® is a registered trademark. Quercustm, and Hessiantm are trademarks of Caucho Technology.

Cloud-optimized Resin Server is a Java EE certified Java Application Server, and Web Server, and Distributed Cache Server (Memcached).
Leading companies worldwide with demand for reliability and high performance web applications including SalesForce.com, CNET, DZone and many more are powered by Resin.

home company blog wiki docs 
app server web server 
health cloud java ee pro