https://www.gravatar.com/avatar/f819a092b23cb25a09f87436f68df217?s=240&d=mp

Ang Gao

Software Engineer @ TripAdvisor, CS Ph.D @ University College Cork

Spark Note 01

Driver A Spark driver (aka an application’s driver process) is a JVM process that hosts SparkContext for a Spark application. It is the master node in a Spark application. It splits a Spark application into tasks and schedules them to run on executors. A driver is where the task scheduler lives and spawns tasks across workers. A driver coordinates workers and overall execution of tasks. It hosts Web UI for the environment.

Scala Note 01

当对一个数取模时,可以等价 a%b=a-a/b*b 1 2 println(-10%3)//-10%3=(-10)-(-3)*3=-10+9=-1 println(-10%-3)//-10%-3=(-10)-(3)*-3=-10+9=-1 在 scala 中支持代码块,返回值 1 2 3 val res = { if (num > 1) "ok" else 100 } Scala 不支持三目运算符 , 在 Scala 中使用 if – else 的方式实现 1 val

Network 02

Network Interfaces Network interfaces represent a point of connection between a computer and a network. interfaces can be software or hardware /proc filesystem /proc/net/[tcp|udp|raw] /proc/net/route /proc/net/dev /proc/net/arp /proc/net/snmp /proc/sys/net which concerns various networking topics /proc/sys/net/core/ message_burst, message_cost message_burst defaults to 10 and message_cost defaults to 5. This means the kernel is limited to logging 10 entries every 5 seconds. netdev_max_backlog maximum number of packets allowed to queue on a particular interface.

Network 01

MAC Media access control (MAC) addresses are identifiers uniquely assigned to network interfaces. first 3 octets refer to the Organizationally Unique Identifier (OUI) which can help identify manufacturers IP IP addresses are means of identifying devices connected to a network under Internet Protocol. In IPv4 we can partition a block of addresses into a subnet. The number that comes after the slash (/) is the subnet mask. This represents how many bits are in the network address, the remaining bits identify a host within the network.

Keeping a fork up to date

Clone your fork: 1 git clone git@github.com:YOUR-USERNAME/YOUR-FORKED-REPO.git Add remote from original repository in your forked repository: 1 2 3 cd into/cloned/fork-repo git remote add upstream git://github.com/ORIGINAL-DEV-USERNAME/REPO-YOU-FORKED-FROM.git git fetch upstream Updating your fork from original repo to keep up with their changes: 1 git pull upstream master

Gatsby - Blazing fast React.js static site generator

Transform plain text and data into dynamic blogs and websites using the latest web technologies. Supports pages written in Markdown, HTML, and React.js out of the box. Easy to add support for additional file types. Leverages React’s component model and React Router’s “nested component hierarchy” to make templating incredibly powerful and intuitive. All templates, css, and content are hot reloadable — powered by webpack. Makes for a brilliant development experience