/** * Will verify that given task tracker is not blacklisted * @param client tasktracker info * @param conf modified configuration object * @param cluster mrcluster instance * @throws IOException thrown if verification fails */ public void verifyTTNotBlackListed(TTClient client, Configuration conf, MRCluster cluster) throws IOException { int interval = conf.getInt("mapred.healthChecker.interval",0); Assert.assertTrue("Interval cannot be zero.",interval != 0); UtilsForTests.waitFor(interval+2000); String defaultHealthScript = conf.get("mapred.healthChecker.script.path"); Assert.assertTrue("Task tracker is not healthy", nodeHealthStatus(client, true) == true); TaskTrackerStatus status = client.getStatus(); JTClient jclient = cluster.getJTClient(); Assert.assertTrue("Failed to move task tracker to healthy list", jclient.getProxy().isBlackListed(status.getTrackerName()) == false); Assert.assertTrue("Health script was not set",defaultHealthScript != null); }
/** * Verifies that the given task tracker is blacklisted * @param conf modified Configuration object * @param client tasktracker info * @param errorMessage that needs to be asserted * @param cluster mr cluster instance * @throws IOException is thrown when verification fails */ public void verifyTTBlackList(Configuration conf, TTClient client, String errorMessage, MRCluster cluster) throws IOException{ int interval = conf.getInt("mapred.healthChecker.interval",0); Assert.assertTrue("Interval cannot be zero.",interval != 0); UtilsForTests.waitFor(interval+2000); //TaskTrackerStatus status = client.getStatus(); Assert.assertTrue("Task tracker was never blacklisted ", nodeHealthStatus(client, false) == true); TaskTrackerStatus status = client.getStatus(); Assert.assertTrue("The custom error message did not appear", status.getHealthStatus().getHealthReport().trim(). equals(errorMessage)); JTClient jClient = cluster.getJTClient(); Assert.assertTrue("Failed to move task tracker to blacklisted list", jClient.getProxy().isBlackListed(status.getTrackerName()) == true); }
/** * The method return true from the task tracker if it is unhealthy/healthy * depending the blacklisted status * @param client the tracker tracker instance * @param health status information. * @return status of task tracker * @throws IOException failed to get the status of task tracker */ public boolean nodeHealthStatus(TTClient client,boolean hStatus) throws IOException { int counter = 0; TaskTrackerStatus status = client.getStatus(); while (counter < 60) { LOG.info("isNodeHealthy "+status.getHealthStatus().isNodeHealthy()); if (status.getHealthStatus().isNodeHealthy() == hStatus) { break; } else { UtilsForTests.waitFor(3000); status = client.getStatus(); Assert.assertNotNull("Task tracker status is null",status); } counter++; } if(counter != 60) { return true; } return false; }
/** {@inheritDoc} */ @Override public float getCpuUsage() { readProcStatFile(); sampleTime = getCurrentTime(); if (lastSampleTime == TaskTrackerStatus.UNAVAILABLE || lastSampleTime > sampleTime) { // lastSampleTime > sampleTime may happen when the system time is changed lastSampleTime = sampleTime; lastCumulativeCpuTime = cumulativeCpuTime; return cpuUsage; } // When lastSampleTime is sufficiently old, update cpuUsage. // Also take a sample of the current time and cumulative CPU time for the // use of the next calculation. final long MINIMUM_UPDATE_INTERVAL = 10 * jiffyLengthInMillis; if (sampleTime > lastSampleTime + MINIMUM_UPDATE_INTERVAL) { cpuUsage = (float)(cumulativeCpuTime - lastCumulativeCpuTime) * 100F / ((float)(sampleTime - lastSampleTime) * getNumProcessors()); lastSampleTime = sampleTime; lastCumulativeCpuTime = cumulativeCpuTime; } return cpuUsage; }
/** * Test parsing /proc/stat and /proc/cpuinfo * @throws IOException */ public void testParsingProcStatAndCpuFile() throws IOException { // Write fake /proc/cpuinfo file. long numProcessors = 8; long cpuFrequencyKHz = 2392781; String fileContent = ""; for (int i = 0; i < numProcessors; i++) { fileContent += String.format(CPUINFO_FORMAT, i, cpuFrequencyKHz / 1000D) + "\n"; } File tempFile = new File(FAKE_CPUFILE); tempFile.deleteOnExit(); FileWriter fWriter = new FileWriter(FAKE_CPUFILE); fWriter.write(fileContent); fWriter.close(); assertEquals(plugin.getNumProcessors(), numProcessors); assertEquals(plugin.getCpuFrequency(), cpuFrequencyKHz); // Write fake /proc/stat file. long uTime = 54972994; long nTime = 188860; long sTime = 19803373; tempFile = new File(FAKE_STATFILE); tempFile.deleteOnExit(); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCumulativeCpuTime(), FAKE_JIFFY_LENGTH * (uTime + nTime + sTime)); assertEquals(plugin.getCpuUsage(), (float)(TaskTrackerStatus.UNAVAILABLE)); // Advance the time and sample again to test the CPU usage calculation uTime += 100L; plugin.advanceTime(200L); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCumulativeCpuTime(), FAKE_JIFFY_LENGTH * (uTime + nTime + sTime)); assertEquals(plugin.getCpuUsage(), 6.25F); // Advance the time and sample again. This time, we call getCpuUsage() only. uTime += 600L; plugin.advanceTime(300L); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCpuUsage(), 25F); // Advance very short period of time (one jiffy length). // In this case, CPU usage should not be updated. uTime += 1L; plugin.advanceTime(1L); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCumulativeCpuTime(), FAKE_JIFFY_LENGTH * (uTime + nTime + sTime)); assertEquals(plugin.getCpuUsage(), 25F); // CPU usage is not updated. }
public TTInfoImpl() { taskTrackerName = ""; status = new TaskTrackerStatus(); }
public TTInfoImpl(String taskTrackerName, TaskTrackerStatus status) { super(); this.taskTrackerName = taskTrackerName; this.status = status; }
@Override public TaskTrackerStatus getStatus() { return status; }
/** * Test parsing /proc/stat and /proc/cpuinfo * @throws IOException */ @Test public void testParsingProcStatAndCpuFile() throws IOException { // Write fake /proc/cpuinfo file. long numProcessors = 8; long cpuFrequencyKHz = 2392781; String fileContent = ""; for (int i = 0; i < numProcessors; i++) { fileContent += String.format(CPUINFO_FORMAT, i, cpuFrequencyKHz / 1000D) + "\n"; } File tempFile = new File(FAKE_CPUFILE); tempFile.deleteOnExit(); FileWriter fWriter = new FileWriter(FAKE_CPUFILE); fWriter.write(fileContent); fWriter.close(); assertEquals(plugin.getNumProcessors(), numProcessors); assertEquals(plugin.getCpuFrequency(), cpuFrequencyKHz); // Write fake /proc/stat file. long uTime = 54972994; long nTime = 188860; long sTime = 19803373; tempFile = new File(FAKE_STATFILE); tempFile.deleteOnExit(); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCumulativeCpuTime(), FAKE_JIFFY_LENGTH * (uTime + nTime + sTime)); assertEquals(plugin.getCpuUsage(), (float)(TaskTrackerStatus.UNAVAILABLE)); // Advance the time and sample again to test the CPU usage calculation uTime += 100L; plugin.advanceTime(200L); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCumulativeCpuTime(), FAKE_JIFFY_LENGTH * (uTime + nTime + sTime)); assertEquals(plugin.getCpuUsage(), 6.25F); // Advance the time and sample again. This time, we call getCpuUsage() only. uTime += 600L; plugin.advanceTime(300L); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCpuUsage(), 25F); // Advance very short period of time (one jiffy length). // In this case, CPU usage should not be updated. uTime += 1L; plugin.advanceTime(1L); updateStatFile(uTime, nTime, sTime); assertEquals(plugin.getCumulativeCpuTime(), FAKE_JIFFY_LENGTH * (uTime + nTime + sTime)); assertEquals(plugin.getCpuUsage(), 25F); // CPU usage is not updated. }
