debug1: identity file /home/anonymous/.ssh/id_rsa type 1 debug1: key_load_public: No such file or directory debug1: identity file /home/anonymous/.ssh/id_rsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/anonymous/.ssh/id_dsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/anonymous/.ssh/id_dsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/anonymous/.ssh/id_ecdsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/anonymous/.ssh/id_ecdsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/anonymous/.ssh/id_ed25519 type -1 debug1: key_load_public: No such file or directory debug1: identity file /home/anonymous/.ssh/id_ed25519-cert type -1
debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_rsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_rsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_dsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_dsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ecdsa type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ecdsa-cert type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ed25519 type -1 debug1: key_load_public: No such file or directory debug1: identity file /root/.ssh/id_ed25519-cert type -1 debug1: Enabling compatibility mode for protocol 2.0
The default behaviour when pressing the PrintScreen key is to automatically place your screenshot in the Pictures folder in your home directory (i.e. “~/Pictures”). The click and the flash mean that the screenshot has been taken, so just check the Pictures folder for your screenshot.
Other than just the “Print Screen’ key, which saves your whole Desktop to the Pictures folder, GNOME3 also has the following shortcuts enabled by default for screenshot actions:
PrintScreen – Takes a screenshot of your entire desktop and saves it to the Pictures folder.
1 2 3 4 5 6
Alt + PrintScreen -- Saves a screenshot of the focused window to the Pictures Folder Shift + PrintScreen -- Lets you select an area of the screen, and saves to the Pictures Folder Ctrl + PrintScreen -- Takes a screenshot of your entire desktop and copies it to the clipboard. Ctrl + Alt + PrintScreen -- copies a screenshot of the focused window to the clipboard. Ctrl + Shift + PrintScreen -- Lets you select an area of the screen, and copies it to the clipboard. Ctrl + Shift + Alt + R -- Records a Screencast) of your entire desktop and saves it to your Videos folder.
set mapred.reduce.tasks=50; insert into table xxx select * from xxx distribute by rand();
**备注:**第一个set设置的就是最后你要生成的文件个数,后面的distribute by rand()保证了记录随机分配到50个文件,不管里数据量有多小,最后这50个文件的大小应该是一致的.
动态分区产生小文件
有些场景会产生大量的文件,比如动态分区插入,或者两个比较大的表做join,对于大表做join,我没有细测,但是我发现我用一个很小的表(大概70M),去join一个很大的表(大概400G),由于Hive在处理小表join大表的时候会做优化,左边的表会都加载到内存里面,然后分发到各个节点和大表做join,这样最后就会在大表所在的节点产生最终的结果,后果就是会原来大表的那些文件现在都变成小文件了,小文件太多其实对性能还是有影响的,这个其实可以最后用一个reducer来合并小文件。 主要说一下动态分区产生小文件问题,这是个很有意思的问题,动态分区好用,但是为啥会产生这么多小文件。原因就在于,假设动态分区初始有N个mapper,那么最后生成了m个分区,最终会有多少个文件生成呢?答案是N*m,是的,每一个mapper会生成m个文件,就是每个分区都会对应一个文件,这样的话你算一下。所以小文件就会成倍的产生。怎么解决这个问题,通常处理方式也是像上面那样,让数据尽量聚到一个reducer里面,因为有时候虽然动态分区不会产生reducer,但是也就意味着最后没有进行文件合并,我们也可以用distribute by rand()这句来保证数据聚类到相同的reducer。
KYLIN_HOME is set to /usr/dev/apache-kylin-1.5.4.1-bin cat:无效选项 -- 1 Try 'cat --help' for more information. -mkdir: Not enough arguments: expected 1 but got 0 Usage: hadoop fs [generic options] -mkdir [-p] <path> ... failed to create , Please make sure the user has right to access
localhost: +======================================================================+ localhost: | Error: JAVA_HOME is not set | localhost: +----------------------------------------------------------------------+ localhost: | Please download the latest Sun JDK from the Sun Java web site | localhost: | > http://www.oracle.com/technetwork/java/javase/downloads | localhost: | | localhost: | HBase requires Java 1.7 or later. | localhost: +======================================================================+ starting master, logging to /usr/dev/hbase-1.2.3/bin/../logs/hbase-anonymous-master-anonymous.out starting regionserver, logging to /usr/dev/hbase-1.2.3/bin/../logs/hbase-anonymous-1-regionserver-anonymous.out