Starlight
From HacDC Wiki
Simple project server, comparable to Shimmer. Increased emphasis on performance and redundancy.
Most likely, this server will continue operating for several years, even as newer and better servers supplant it.
For discussion purposes, server is nicknamed "Starlight".
ChiefAdmins
- Julia "juri" Longtin
- Matthew "mirage335" Hines
Support
#hacdc on irc.freenode.net (or via chat.hacdc.org)
Login
On any unix terminal emulator, enter the command:
ssh <userName>@starlight.srv.hacdc.org -p ####
On other platforms, try applications like PuTTY with similar settings. Local connections (within HacDC's network) will need to use IP address 192.168.14.254 .
Recommendations
- Set restrictive permissions on directories belonging to you.
- Create a directory for yourself on the fast array at "/mnt/fast/users".
- Use fast array for I/O intensive active jobs (eg. compilation, statistics, simulation). Fast array provides maximum speed, theoretically sustaining upwards of 900MB/s or 1000 IOPS.
- Use root filesystem (eg. "/home", "/var/www/html") for lightweight services including important data (eg. MediaWiki, git, email). Root filesystem provides maximum durability, surviving 2/6 disk failures.
- Periodically backup (rsync) important data from fast array to root filesystem.
- Backup especially critical data (eg. databases) offsite (eg. dropbox).
HacDC admins are happy to help, and may be able to provide assistance with other services (eg. offsite backup).
CommandReference
Routinely important commands unique to this server documented as follows.
Users
List
List OpenVZ VMs created for you.
cat ~/hovzList
Startup
sudo /usr/sbin/vzctl start <name>
Shutdown
sudo /usr/sbin/vzctl stop <name>
Console
Opens command line of running VM <name> .
sudo /usr/sbin/vzctl enter <name>
Root (Admin)
hostedOpenVZ
- ./hovzNewUser <userName> # Creates user account with default OpenVZ VM and permissions.
- ./hovzDelUser <userName> # Deletes user account, associated VMs, and associated OpenVZ permissions.
- ./hovzNewVM <userName> <VM_Number> <dist> # Creates VM accessible to non-root user <userName> . Dist is often"debian-7.0-x86_64-minimal" .
- vzctl set CTID --ram 128M --swap 256M --save
- vzctl set CTID --diskspace 4G:4.2G --save
Autostart
Through normal reboots, host will suspend/resume any guest VMs.
iptables
Investigation
- iptables -t nat -L -n -v #Lists port forwarding rules.
- iptables -t nat -F #Deletes port forwarding rules.
- iptables -D PREROUTING 1 #Deletes port forwarding rule.
Forwarding
Specific
/sbin/iptables -t nat -A PREROUTING -p tcp -i eth0 --dport 20283 -j DNAT --to 10.174.10.5:22
Persistent
Package iptables-persistent has been installed. Upon installation, existing iptables rules were saved. Edit /etc/iptables/rules.v4 .
ReInstall
- Install OpenVZ, documented at http://openvz.org/Installation_on_Debian .
- Install hostedOpenVZ, documented at https://github.com/mirage335/hostedOpenVZ .
- Change default kernel in /etc/default/grub.cfg .
- Apply "iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE" .
Characteristics
- CPU - Dual Processor, Octo-Core, Intel Xeon X5355
- RAM - 48GB ECC
- HDD Bulk Array - 20TB RAID6
- HDD Fast Array - 5TB RAID0, Estimated 900MB/s, 1000 IOPS.
- Hostname - HacDC-shared-m335-2
- FQDN - HacDC-shared-m335-2.srv.hacdc.org
Redundancy (RAID)
Root filesystem is comprised of six WD60EZRX (WD Green 6TB) disks in a RAID6 (dual-parity array). Disks were intentionally purchased from diverse sources. Data will not be lost unless more than two drives in this array fail simultaneously.
Fast filesystem has worse than no redundancy, being a RAID0 stripe across the front of each disk.
Internal backups and some spare hardware are kept to minimize potential downtime. Disk failures are generally expected to be repaired without shutdown.
Policies
Permissions
Root
HacDC members with a demonstrable need or willingness to manage non-root users may be given root access. Imperative that root users do not jeopardize uptime.
Shell
All HacDC members are welcome to non-root shell accounts, directly provided by the server.
OpenVZ
All HacDC members are welcome to OpenVZ Virtual Machines. Resources, including CPU, disk space, and external network ports, will be allocated on an as-needed first-come-first-serve basis.
Root users, please use the provided hostedOpenVZ scripts. Following the naming conventions set therein helps account for which resources belong to whom.
Notifications
Internal server email will notify users, if feasible, on the following schedules, subject to change.
- Three days before planned downtime exceeding one hour.
- One week before planned permanent downtime (obsolescence).
Forwarding to another email address is possible with the following command.
echo '[email protected]' > ~/.forward
Removal
- Up to three months before removal of ex-member accounts. Exceptions on a case-by-case basis.
- Compromised accounts will be immediately removed or disabled as discovered.
- Excessive resource use, including disk space consumption, may be terminated, particularly if account holder is not responsive to inquiries.
DataLoss
- Failure of more than two disks will result in unrecoverable loss of data on this server.
- Failure of any disk on fast array will result in unrecoverable loss of data from that partition.
- Removed accounts or terminated services may result in immediate, permanent deletion.
- As array repair procedures are rarely done, catastrophic mistakes are possible.
- Virtual Machines are stored on fast array by default, periodically backed up internally.
Privacy
Machine is security is not especially tight. Sysadmins may investigate possible abuse. Privacy may exist, but is not a reasonable expectation.
AcceptableUse
Disk
Vast quantities of disk space are available. Still, please be considerate, and understand that on-mission HacDC activities may be given priority.
Sharing
Sharing of account resources is permitted, however, additional resources will be allocated according to individual member needs for specific purposes. As a reminder, compromised accounts will be immediately removed or disabled as discovered.
Bandwidth
Although HacDC has a high-capacity 175/15Mbit link, it is shared with other tenants. Please be courteous. If internet bandwidth use is required for downloading large files, between 2200EDT and 0600EDT is best.
Hard bandwidth limits have not been set to ease administration of local file servers. However, please configure applications to limit maximum bandwidth use where possible, particularly for web servers.
- NOTE: If your bandwidth use is too high during normal hours, the church administrator will cut bandwidth available to the whole subnet and will call mirage335, juri_, Ethan Waldo, or other admin team members to resolve the issue (no one will be happy).
Special Services
HTTP/HTTPs
Ports 80/443 are available through reverse proxy. Please setup a relevant domain name (eg. *.member.hacdc.org) through FreeDNS or other service, and contact the HacDC admin team for assistance.
NamingConvention
Preferred network names for participating machines and virtual machines.
Physical
Hostnames
HacDC-shared-m335-2
- HacDC - Identifies machine as HacDC or HacDC member property on the local network.
- shared - One of private, restricted, shared. Private = one member only (eg. RasPi). Restricted = limited access (eg. HacDC web server). Shared = shared resources (ie. remote shells).
- m335 - Abbreviated member name or screen name. Full contact details should be written on the physical machine.
- 1 - Unique identifier.
FQDNs
HacDC-shared-m335-2.router.hacdc.org
Virtual
Hostnames
mirage335-1
- mirage335 - Local username.
- 1 - Unique machine identifier.
FQDNs
mirage335-1.HacDC-shared-m335-2.router.hacdc.org
Timezone
EST/EDT aka US Eastern, America/NewYork, or "Eastern"
Workarounds
SSH KeepAlive
Due to limited bandwidth, SSH sessions may freeze. To prevent this, consider the following to configuration, appended to client-side ~/.ssh/config .
Host * ServerAliveInterval 30
Disclaimer
No guarantees. Admins are expected to make best efforts toward reliability, security, and privacy. Monitoring may include without limitation system health, resource consumption, and authentication failures, and the like. More intrusive monitoring, such as of user filesystems, should be avoided if possible.
Excepting the case, reasonably requsitioned as a supply, the server is property of Julia Longtin. Administration is at the discretion of her and designated admins.
Credits
Julia Longtin - Donated core hardware.