In file included from /usr/local/osquery/include/thrift/TDispatchProcessor.h:22: In file included from /tmp/yaourt-tmp-bverdier/aur-osquery-git/src/osquery-git/generated/gen-cpp/Extension.h:10: In file included from /tmp/yaourt-tmp-bverdier/aur-osquery-git/src/osquery-git/generated/gen-cpp/Extension.cpp:7: I'm getting build errors when trying to build from the latest version: Make: Leaving directory '/home/viq/.cache/pacaur/osquery-git/src/osquery-git'
Libosquery_additional.a():(._ZTIN7osquery17GlogRocksDBLoggerE+0x10): undefined reference to `typeinfo for rocksdb::Logger'Ĭollect2: error: ld returned 1 exit status usr/bin/g++ -std=c++14 -Wl,-O1,-sort-common,-as-needed,-z,relro,-z,now -rdynamic CMakeFiles/daemon.dir/devtools/ CMakeFiles/daemon.dir/main/ CMakeFiles/daemon.dir/main/posix/ -o osqueryd -Wl,-whole-archive libosquery.a -Wl,-no-whole-archive -Wl,-whole-archive libosquery_additional.a -Wl,-no-whole-archive -lz -Wl,-Bdynamic -lgflags -Wl,-Bdynamic -lthrift -ldl -Wl,-Bstatic -lboost_system -lboost_filesystem -lboost_thread -lboost_context -Wl,-Bdynamic -lrt -lc -lglog -rdynamic -Wl,-zrelro -Wl,-znow -pie -Wl,-build-id -static-libstdc++ -ludev -laudit -larchive -lzstd -lrdkafka -Wl,-Bstatic -laws-cpp-sdk-kinesis -laws-cpp-sdk-firehose -laws-cpp-sdk-core -laws-cpp-sdk-sts -Wl,-Bdynamic -lresolv -lcryptsetup -ldevmapper -llvm2app -lgcrypt -lgpg-error -lblkid -lip4tc -Wl,-Bstatic -ldpkg -Wl,-Bdynamic -llzma -lrpm -lrpmio -Wl,-Bstatic -lpopt -Wl,-Bdynamic -ldb -laugeas -lfa -lxml2 -Wl,-Bstatic -laws-cpp-sdk-ec2 -Wl,-Bdynamic -ltsk -lyara -llldpctl -lrocksdb_lite -Wl,-Bstatic -lboost_regex -Wl,-Bdynamic -lssl -lcrypto -lpthread -lmagic -lbz2 -luuid Building osqueryd: /home/viq/.cache/pacaur/osquery-git/src/osquery-git/osquery/osquerydĬd /home/viq/.cache/pacaur/osquery-git/src/osquery-git/osquery & /usr/bin/cmake -E cmake_link_script CMakeFiles/daemon.dir/link.txt -verbose=1
#Osquery kills ec2 upgrade#
upgrade to a larger instance (the m4.2xlarge also has 8 vCPUs but 32 GiB of RAM, although it’s a General Purpose rather than a Compute Optimised instance) or.These are the options I’m exploring to solve the problem: I thought that the available RAM and swap were sufficient, but evidently I was mistaken. Out of memory: Kill process 3026 (R) score 649 or sacrifice child Then the final coup de grâce: killing PID 3026, which in turn took down the rest of the R tasks. The biggest memory hog is PID 3026, which is the R master task. Note the final eight lines, which correspond to my optimisation job (it’s running in parallel with seven worker threads). uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name Then some details on memory allocation to individual processes. Note that all of the swap space has been used! Some high level information on memory allocation.
The OOM Killer is responsible for killing tasks when the system is running out of memory. That’s the first sign that something is going horribly wrong: R invoked oom-killer. Slab_reclaimable:8631 slab_unreclaimable:7256 Unevictable:915 dirty:0 writeback:0 unstable:0 Hardware name: Xen HVM domU, BIOS 4.2.amazon R invoked oom-killer: gfp_mask=0x24280ca, order=0, oom_score_adj=0 I’ve edited out some of the irrelevant details. I had a look at the output from dmesg and that immediately pointed me to the source of the problem. Luckily Ubuntu comes with a plethora of tools for debugging. Hard to figure out the source of the problem.
The jobs died with a curt and rather uninformative message in the console: Killed I had the job running on a c4.2xlarge instance with 8 vCPUs and 15 GiB of RAM. This is what I found and how I am resolving the problem. Definitely not a coincidence! So I investigated. However, this morning it was killed again. I shrugged it off as an anomaly and restarted the job. I’ve got a long running optimisation problem on a EC2 instance.
0 kommentar(er)
