diff options
author | Thanos Doukoudakis <thanos.doukoudakis@isode.com> | 2017-06-16 10:45:03 (GMT) |
---|---|---|
committer | Thanos Doukoudakis <thanos.doukoudakis@isode.com> | 2017-07-07 13:47:58 (GMT) |
commit | 1b678a155adca8957866e016b0e344eee6290466 (patch) | |
tree | 08af7836ed7fca39a61f85941ec7545be7455c97 | |
parent | 2a251161c5baa688ec12068346359bd829ab2ea1 (diff) | |
download | swift-1b678a155adca8957866e016b0e344eee6290466.zip swift-1b678a155adca8957866e016b0e344eee6290466.tar.bz2 |
Make gtest output more compact
This patch will make the checker test application output to use a custom
printer for google test results. The output will be less verbose, and
similar to the output of CppUnit. If the --verbose flag is used, the
default printer will be used instead.
Test-information:
Tested on Windows 10 and Ubuntu 16.04.
Change-Id: I1488cf576ab07da03b0dfcc93a48a8518d5afc06
-rw-r--r-- | COPYING.thirdparty | 42 | ||||
-rw-r--r-- | QA/Checker/CppUnitTestResultPrinter.cpp | 142 | ||||
-rw-r--r-- | QA/Checker/CppUnitTestResultPrinter.h | 58 | ||||
-rw-r--r-- | QA/Checker/SConscript | 2 | ||||
-rw-r--r-- | QA/Checker/checker.cpp | 7 |
5 files changed, 249 insertions, 2 deletions
diff --git a/COPYING.thirdparty b/COPYING.thirdparty index 0134aaa..35cad47 100644 --- a/COPYING.thirdparty +++ b/COPYING.thirdparty @@ -70,61 +70,61 @@ No term or provision of this License shall be deemed waived and no breach consen This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You. The rights granted under, and the subject matter referenced, in this License were drafted utilizing the terminology of the Berne Convention for the Protection of Literary and Artistic Works (as amended on September 28, 1979), the Rome Convention of 1961, the WIPO Copyright Treaty of 1996, the WIPO Performances and Phonograms Treaty of 1996 and the Universal Copyright Convention (as revised on July 24, 1971). These rights and subject matter take effect in the relevant jurisdiction in which the License terms are sought to be enforced according to the corresponding provisions of the implementation of those treaty provisions in the applicable national law. If the standard suite of rights granted under applicable copyright law includes additional rights not granted under this License, such additional rights are deemed to be included in the License; this License is not intended to restrict the license of any rights under applicable law. --- END OF CREATE COMMONS ATTRIBUTION 3.0 LICENSE --- ========= Lato Font ========= Copyright (c) 2010-2015, Ćukasz Dziedzic (dziedzic@typoland.com), with Reserved Font Name Lato. This Font Software is licensed under the SIL Open Font License, Version 1.1. This license is copied below, and is also available with a FAQ at: http://scripts.sil.org/OFL ----------------------------------------------------------- SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007 ----------------------------------------------------------- PREAMBLE The goals of the Open Font License (OFL) are to stimulate worldwide development of collaborative font projects, to support the font creation efforts of academic and linguistic communities, and to provide a free and open framework in which fonts may be shared and improved in partnership with others. The OFL allows the licensed fonts to be used, studied, modified and redistributed freely as long as they are not sold by themselves. The -fonts, including any derivative works, can be bundled, embedded, +fonts, including any derivative works, can be bundled, embedded, redistributed and/or sold with any software provided that any reserved names are not used by derivative works. The fonts and derivatives, however, cannot be released under any other type of license. The requirement for fonts to remain under this license does not apply to any document created using the fonts or their derivatives. DEFINITIONS "Font Software" refers to the set of files released by the Copyright Holder(s) under this license and clearly marked as such. This may include source files, build scripts and documentation. "Reserved Font Name" refers to any names specified as such after the copyright statement(s). "Original Version" refers to the collection of Font Software components as distributed by the Copyright Holder(s). "Modified Version" refers to any derivative made by adding to, deleting, or substituting -- in part or in whole -- any of the components of the Original Version, by changing formats or by porting the Font Software to a new environment. "Author" refers to any designer, engineer, programmer, technical writer or other person who contributed to the Font Software. PERMISSION & CONDITIONS Permission is hereby granted, free of charge, to any person obtaining a copy of the Font Software, to use, study, copy, merge, embed, modify, redistribute, and sell modified and unmodified copies of the Font Software, subject to the following conditions: @@ -235,44 +235,84 @@ which incorporates getifaddrs() code under the following license: --- START OF SOLARIS getifaddrs() LICENSE Copyright (c) 2006 WIDE Project. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the project nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE PROJECT AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE PROJECT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. --- END OF SOLARIS getifaddrs() LICENSE +=========== +Google Test +=========== + +The checker application uses QA\Checker\CppUnitTestResultPrinter.cpp file +which incorporates PrettyUnitTestResultPrinter class from gtest.cc under the following license: + +// Copyright 2005, Google Inc. +// All rights reserved. +// +// Redistribution and use in source and binary forms, with or without +// modification, are permitted provided that the following conditions are +// met: +// +// * Redistributions of source code must retain the above copyright +// notice, this list of conditions and the following disclaimer. +// * Redistributions in binary form must reproduce the above +// copyright notice, this list of conditions and the following disclaimer +// in the documentation and/or other materials provided with the +// distribution. +// * Neither the name of Google Inc. nor the names of its +// contributors may be used to endorse or promote products derived from +// this software without specific prior written permission. +// +// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +// +// Author: wan@google.com (Zhanyong Wan) +// +// The Google C++ Testing Framework (Google Test) + ========= Trolltech ========= Demo code from http://git.forwardbias.in/?p=lineeditclearbutton.git with license: /**************************************************************************** ** ** Copyright (c) 2007 Trolltech ASA <info@trolltech.com> ** ** Use, modification and distribution is allowed without limitation, ** warranty, liability or support of any kind. ** ****************************************************************************/ diff --git a/QA/Checker/CppUnitTestResultPrinter.cpp b/QA/Checker/CppUnitTestResultPrinter.cpp new file mode 100644 index 0000000..475411b --- /dev/null +++ b/QA/Checker/CppUnitTestResultPrinter.cpp @@ -0,0 +1,142 @@ +/* + * Copyright (c) 2017 Isode Limited. + * All rights reserved. + * See the COPYING file for more information. + */ + + // Copyright 2005, Google Inc. + // All rights reserved. + // + // Redistribution and use in source and binary forms, with or without + // modification, are permitted provided that the following conditions are + // met: + // + // * Redistributions of source code must retain the above copyright + // notice, this list of conditions and the following disclaimer. + // * Redistributions in binary form must reproduce the above + // copyright notice, this list of conditions and the following disclaimer + // in the documentation and/or other materials provided with the + // distribution. + // * Neither the name of Google Inc. nor the names of its + // contributors may be used to endorse or promote products derived from + // this software without specific prior written permission. + // + // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + // + // Author: wan@google.com (Zhanyong Wan) + // + // The Google C++ Testing Framework (Google Test) + +#include <QA/Checker/CppUnitTestResultPrinter.h> + +namespace testing { + + namespace internal { + enum GTestColor { + COLOR_DEFAULT, + COLOR_RED, + COLOR_GREEN, + COLOR_YELLOW + }; + void ColoredPrintf(GTestColor color, const char* fmt, ...); + } + +using namespace internal; + +static std::string PrintTestPartResultToString(const TestPartResult& testPartResult) { + return (Message() << internal::FormatFileLocation(testPartResult.file_name(), testPartResult.line_number()) + << " " << (testPartResult.type() == TestPartResult::Type::kSuccess ? "Success" : "Failure") << std::endl << testPartResult.message()).GetString(); +} + +void CppUnitTestResultPrinter::OnTestIterationStart(const UnitTest& unitTest, int iteration) { + if (GTEST_FLAG(repeat) != 1) { + std::cout << std::endl << "Repeating all tests (iteration" << iteration + 1 << ") . . ." << std::endl << std::endl; + } + const char* const filter = GTEST_FLAG(filter).c_str(); + if (!String::CStringEquals(filter, "*")) { + ColoredPrintf(COLOR_YELLOW, "Note: %s filter = %s\n", GTEST_NAME_, filter); + } + if (GTEST_FLAG(shuffle)) { + ColoredPrintf(COLOR_YELLOW, "Note: Randomizing tests' orders with a seed of %d .\n", unitTest.random_seed()); + } + ColoredPrintf(COLOR_GREEN, "[==========] "); + std::cout << "Running " << unitTest.test_to_run_count() << " test(s) from " << unitTest.test_case_to_run_count() << " test case(s)" << std::endl; + fflush(stdout); +} + +void CppUnitTestResultPrinter::OnTestPartResult(const TestPartResult& testPartResult) { + if (testPartResult.failed()) { + assertionsFailures_ << PrintTestPartResultToString(testPartResult) << std::endl; + } +} + +void CppUnitTestResultPrinter::OnTestEnd(const TestInfo& testInfo) { + if (testInfo.result()->Passed()) { + ColoredPrintf(COLOR_GREEN, "."); + } + else { + ColoredPrintf(COLOR_RED, "F"); + testFailures_ << std::endl << testInfo.name() << "." << testInfo.test_case_name() << std::endl << assertionsFailures_.rdbuf(); + assertionsFailures_.clear(); + } +} + +void CppUnitTestResultPrinter::OnTestIterationEnd(const UnitTest& unitTest, int) { + ColoredPrintf(COLOR_GREEN, "\n[==========] "); + std::cout << unitTest.test_to_run_count() << " test(s) from " << unitTest.test_case_to_run_count() << " test case(s) ran. "; + if (GTEST_FLAG(print_time)) { + std::cout << "(" << StreamableToString(unitTest.elapsed_time()).c_str() << " ms total)"; + } + ColoredPrintf(COLOR_GREEN, "\n[ PASSED ] "); + std::cout << unitTest.successful_test_count() << std::endl; + + int num_failures = unitTest.failed_test_count(); + if (!unitTest.Passed()) { + ColoredPrintf(COLOR_RED, "[ FAILED ] "); + std::cout << unitTest.failed_test_count() << " test(s), listed below:" << std::endl; + PrintFailedTests(unitTest); + std::cout << std::endl << num_failures << " FAILED " << ((num_failures == 1) ? "TEST" : "TESTS") << std::endl; + } + int num_disabled = unitTest.reportable_disabled_test_count(); + if (num_disabled && !GTEST_FLAG(also_run_disabled_tests)) { + if (!num_failures) { + std::cout << std::endl; + } + std::cout << num_disabled << " test(s) were disabled." << std::endl << std::endl; + } + std::cout << testFailures_.rdbuf(); + fflush(stdout); +} + +// Internal helper for printing the list of failed tests. +void CppUnitTestResultPrinter::PrintFailedTests(const UnitTest& unitTest) { + if (unitTest.failed_test_count() == 0) { + return; + } + for (int i = 0; i < unitTest.total_test_case_count(); ++i) { + const TestCase& testCase = *unitTest.GetTestCase(i); + if (!testCase.should_run() || (testCase.failed_test_count() == 0)) { + continue; + } + for (int j = 0; j < testCase.total_test_count(); ++j) { + const TestInfo& testInfo = *testCase.GetTestInfo(j); + if (!testInfo.should_run() || testInfo.result()->Passed()) { + continue; + } + ColoredPrintf(COLOR_RED, "[ FAILED ] "); + std::cout << testCase.name() << "." << testInfo.name() << std::endl; + } + } +} + +} diff --git a/QA/Checker/CppUnitTestResultPrinter.h b/QA/Checker/CppUnitTestResultPrinter.h new file mode 100644 index 0000000..8e09243 --- /dev/null +++ b/QA/Checker/CppUnitTestResultPrinter.h @@ -0,0 +1,58 @@ +/* + * Copyright (c) 2017 Isode Limited. + * All rights reserved. + * See the COPYING file for more information. + */ + + // Copyright 2005, Google Inc. + // All rights reserved. + // + // Redistribution and use in source and binary forms, with or without + // modification, are permitted provided that the following conditions are + // met: + // + // * Redistributions of source code must retain the above copyright + // notice, this list of conditions and the following disclaimer. + // * Redistributions in binary form must reproduce the above + // copyright notice, this list of conditions and the following disclaimer + // in the documentation and/or other materials provided with the + // distribution. + // * Neither the name of Google Inc. nor the names of its + // contributors may be used to endorse or promote products derived from + // this software without specific prior written permission. + // + // THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + // "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + // LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + // A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + // OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + // SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + // LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + // DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + // THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + // (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + // OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + // + // Author: wan@google.com (Zhanyong Wan) + // + // The Google C++ Testing Framework (Google Test) + +#pragma once + +#include <gtest/gtest.h> + +namespace testing { + + //CppUnitTestResultPrinter is based on PrettyUnitTestResultPrinter from the google test suite + class CppUnitTestResultPrinter : public ::testing::EmptyTestEventListener { + public: + virtual void OnTestIterationStart(const ::testing::UnitTest& unitTest, int iteration); + virtual void OnTestPartResult(const ::testing::TestPartResult& testPartResult); + virtual void OnTestEnd(const ::testing::TestInfo& test_info); + virtual void OnTestIterationEnd(const ::testing::UnitTest& unitTest, int); + private: + static void PrintFailedTests(const ::testing::UnitTest& unitTest); + std::stringstream assertionsFailures_; + std::stringstream testFailures_; + }; +} diff --git a/QA/Checker/SConscript b/QA/Checker/SConscript index a41a5f9..70458f0 100644 --- a/QA/Checker/SConscript +++ b/QA/Checker/SConscript @@ -1,24 +1,24 @@ import os Import("env") if env["TEST"] : if env["SCONS_STAGE"] == "flags" : env["CHECKER_FLAGS"] = { "LIBS": ["Checker"], "LIBPATH": [Dir(".")], "LINKFLAGS": env["PLATFORM"] == "win32" and ["/SUBSYSTEM:CONSOLE"] or [] } if os.path.basename(env["CC"]).startswith(("clang", "gcc")): env["CHECKER_FLAGS"]["CPPFLAGS"] = ["-isystem" + Dir("#/3rdParty/HippoMocks").abspath] else : env["CHECKER_FLAGS"]["CPPPATH"] = ["#/3rdParty/HippoMocks"] if env["SCONS_STAGE"] == "build" : checker_env = env.Clone() checker_env.UseFlags(env["SWIFTEN_FLAGS"]) checker_env.UseFlags(env["BOOST_FLAGS"]) checker_env.UseFlags(env["CPPUNIT_FLAGS"]) checker_env.UseFlags(env["GOOGLETEST_FLAGS"]) - checker_env.Library("Checker", ["checker.cpp", "IO.cpp"]) + checker_env.Library("Checker", ["checker.cpp", "IO.cpp", "CppUnitTestResultPrinter.cpp"]) diff --git a/QA/Checker/checker.cpp b/QA/Checker/checker.cpp index bcf0f5c..3cc7713 100644 --- a/QA/Checker/checker.cpp +++ b/QA/Checker/checker.cpp @@ -1,42 +1,43 @@ /* * Copyright (c) 2010-2017 Isode Limited. * All rights reserved. * See the COPYING file for more information. */ #include <cstdlib> #include <fstream> #include <string> #include <sstream> #include <gtest/gtest.h> +#include <QA/Checker/CppUnitTestResultPrinter.h> #include <cppunit/BriefTestProgressListener.h> #include <cppunit/TextOutputter.h> #include <cppunit/TextTestProgressListener.h> #include <cppunit/TextTestResult.h> #include <cppunit/XmlOutputter.h> #include <cppunit/extensions/TestFactoryRegistry.h> #include <cppunit/ui/text/TestRunner.h> #include <Swiften/Base/Log.h> using Swift::Log; int main(int argc, char* argv[]) { bool verbose = false; bool outputXML = false; Log::setLogLevel(Swift::Log::error); // Parse parameters std::vector<std::string> testsToRun; for (int i = 1; i < argc; ++i) { std::string param(argv[i]); if (param == "--verbose") { verbose = true; } else if (param == "--xml") { outputXML = true; } else if (param == "--debug") { @@ -99,40 +100,46 @@ int main(int argc, char* argv[]) { for (std::vector<std::string>::const_iterator i = testsToRun.begin(); i != testsToRun.end(); ++i) { try { runner.run(controller, *i); } catch (const std::exception& e) { std::cerr << "Error: " << e.what() << std::endl; return -1; } } // Output the results if (outputXML) { std::ofstream cppUnitXUnitOutputFile; cppUnitXUnitOutputFile.open(cppunitOutputFilename, std::ofstream::out | std::ofstream::trunc); if (cppUnitXUnitOutputFile.is_open()) { CppUnit::XmlOutputter outputter(&result, cppUnitXUnitOutputFile); outputter.write(); } else { std::cerr << "Failed to overwrite " << cppunitOutputFilename << " output file." << std::endl; return 1; } } else { CppUnit::TextOutputter outputter(&result, std::cerr); outputter.write(); } auto googleTestWasSuccessful = false; try { + if (!verbose) { + testing::UnitTest& unitTest = *testing::UnitTest::GetInstance(); + testing::TestEventListeners& listeners = unitTest.listeners(); + delete listeners.Release(listeners.default_result_printer()); + listeners.Append(new testing::CppUnitTestResultPrinter); + } googleTestWasSuccessful = RUN_ALL_TESTS() == 0 ? true : false; } catch (const ::testing::internal::GoogleTestFailureException& e) { googleTestWasSuccessful = false; SWIFT_LOG(error) << "GoogleTestFailureException was thrown: " << e.what() << std::endl; } auto cppUnitWasSuccessful = result.wasSuccessful() ? true : false; return (googleTestWasSuccessful && cppUnitWasSuccessful) ? 0 : 1; } |