tldr gnu

  the core GNU packages commonly found in the base system include:

1. coreutils: Provides essential commands for file and text manipulation, including ls, cp, mv, rm, mkdir, rmdir, cat, echo, touch, chown, chgrp, and many more.

2. findutils: Includes the find command for searching and locating files and directories in a directory hierarchy.

3. diffutils: Contains tools for comparing and finding differences between files, including the diff and cmp commands.

4. grep: The GNU version of the grep command for searching text files using regular expressions.

5. sed: A stream editor for text transformations.

6. awk: A versatile text processing tool for manipulating data.

7. tar: A utility for creating and extracting archives.

8. gzip/gunzip: Provides file compression and decompression utilities.

9. bash: The GNU Bourne-Again Shell, a command-line interpreter that is commonly used as the default system shell.

10. file: A command for determining file types.

11. xargs: A utility for building and executing command lines from standard input.

12. diff: A program for comparing files and showing their differences.

13. patch: A utility for applying differences between files, often used for source code patching.

14. ncurses: The library that provides text-based user interfaces for terminal applications.

15. gawk: The GNU implementation of AWK, a powerful text processing language.

16. bison: A parser generator used for creating parsers and compilers.

17. flex: A lexical analyzer generator used for creating scanners for programming languages.

18. coreutils: A package that provides various fundamental commands for file and text manipulation.

These packages and utilities form the core set of tools that are included in most GNU/Linux distributions and are considered essential for basic system administration, scripting, and day-to-day tasks. The availability of these utilities ensures that the system can perform fundamental file and text operations, and they are often required for system bootstrapping and recovery.

 

 

 coreutils

 

1. ls: List files and directories in a directory.

2. cd: Change the current directory.

3. pwd: Print the current working directory.

4. touch: Create an empty file or update a file's access and modification timestamps.

5. mkdir: Create directories.

6. rm: Remove files or directories.

7. cp: Copy files and directories.

8. mv: Move or rename files and directories.

9. cat: Concatenate and display file content.

10. more/less: View text files one page at a time.

11. head/tail: Display the beginning or end of a file.

12. grep: Search for patterns in text using regular expressions.

13. find: Search for files and directories in a directory hierarchy.

14. chmod: Change file permissions.

15. chown: Change file ownership.

16. chgrp: Change file group ownership.

17. ln: Create hard or symbolic links.

18. ps: Display information about processes.

19. kill: Terminate processes.

20. top/htop: Monitor system processes and resource usage.

21. df: Display disk space usage.

22. du: Estimate file and directory space usage.

23. tar: Create and extract archives.

24. gzip/gunzip: Compress and decompress files.

25. awk: Text processing tool for manipulating data.

26. sed: Stream editor for text transformations.

27. sort: Sort lines of text files.

28. cut: Remove sections from lines of files.

29. xargs: Build and execute command lines from standard input.

30. tee: Read from standard input and write to standard output and files.

31. date: Display or set the system date and time.

32. echo: Display text to the terminal.

33. basename: Remove directory and suffix from file names.

34. dirname: Remove non-directory suffix from file names.

35. rev: Reverse lines of files.

36. tr: Translate characters.

37. wc: Count lines, words, and characters in files.

38. cmp: Compare two files byte by byte.

39. comm: Compare two sorted files line by line.

40. diff: Compare and find differences between files.

41. ln: Create hard or symbolic links.

42. touch: Change file timestamps.

43. mktemp: Create a temporary file or directory.

 

 

 awk

AWK is a versatile text processing tool that allows you to manipulate data in text files. Here's a quick AWK tutorial to get you started:

1. Basic Structure:

   • AWK programs consist of a pattern-action pair. It processes each line in a file, matching lines that meet the specified pattern and performing the associated action.

2. Printing:

   • The simplest AWK program is awk '{print $0}' file.txt, which prints every line of file.txt.

3. Field Separators:

   • AWK splits lines into fields. By default, fields are separated by spaces or tabs, but you can specify a custom separator with -F or FS.

4. Printing Fields:

   • To print specific fields, use $1, $2, etc. For example, awk '{print $2, $1}' file.txt swaps the first and second fields.

5. Patterns and Conditions:

   • You can add conditions to filter lines. For example, awk '$3 > 10' file.txt prints lines where the third field is greater than 10.

6. Built-in Variables:

   • AWK provides several built-in variables like NF (number of fields) and NR (record number). For example, awk 'NF > 3' file.txt prints lines with more than 3 fields.

7. Regular Expressions:

   • AWK supports regular expressions. Use /pattern/ to match lines with a specific pattern. For example, awk '/error/' file.txt prints lines containing "error."

8. User-Defined Variables:

   • You can create your own variables in AWK. For example, awk '{total += $1} END {print total}' file.txt calculates the sum of the first field.

9. Control Structures:

   • AWK has basic control structures like if, else, and for. For instance, you can use an if statement to conditionally perform actions.

10. Functions:

    • AWK supports functions. You can use built-in functions like split() or create custom functions to perform complex operations.

11. File Output:

    • You can redirect the output to a file using > or >>. For example, awk '/pattern/' file.txt > output.txt saves lines with "pattern" to a new file.

12. Pipelines:

    • AWK can be used in pipelines with other Unix utilities like grep, sort, and sed to perform more complex text processing tasks.

13. One-Liners:

    • AWK is often used for quick one-liners. For example, ls -l | awk '{total += $5} END {print "Total:", total}' calculates the total size of files in the current directory.

AWK is a powerful tool for text processing and data extraction. You can create complex data processing scripts or use it for simple text manipulation tasks. The more you practice and explore its capabilities, the more proficient you'll become.

 

sed 

1. Basic sed Command Structure:

   • sed 's/pattern/replacement/' file replaces occurrences of pattern with replacement in the specified file.

2. Printing Text:

   • To print the contents of a file, use sed -n 'p' file.

3. Substitution:

   • Replace text using the s command. For example, sed 's/old/new/' file replaces the first occurrence of old with new on each line.

4. Global Substitution:

   • Use the g flag to replace all occurrences in a line: sed 's/old/new/g' file.

5. In-Place Editing:

   • To edit a file in-place, use the -i option: sed -i 's/old/new/g' file.

6. Multiple Operations:

   • You can combine multiple operations by separating them with a semicolon: sed 's/one/two/; s/three/four/' file.

7. Print Specific Lines:

   • To print specific lines, use line addresses. For example, sed -n '1,5p' file prints lines 1 to 5.

8. Delete Lines:

   • To delete lines, use the d command. For instance, sed '/pattern/d' file deletes lines containing pattern.

9. Append Text:

   • To append text after a line, use the a command: sed '/pattern/a text to append' file.

10. Insert Text:

    • To insert text before a line, use the i command: sed '/pattern/i text to insert' file.

11. Find and Replace:

    • Replace text on specific lines. For example, sed '3,7s/old/new/g' file replaces old with new on lines 3 to 7.

12. Regular Expressions:

    • sed supports regular expressions. You can use sed with regex patterns to match complex text.

13. Backreferences:

    • Use backreferences to refer to parts of the matched pattern. For example, sed 's/\(first\).*/\1/' file keeps only "first."

14. Save to New File:

    • To save the edited content to a new file, redirect the output: sed 's/old/new/' input.txt > output.txt.

15. Case Insensitivity:

    • Make your search and replace case-insensitive with the i flag: sed 's/pattern/replacement/i' file.

16. Multiple Input Files:

    • You can process multiple files by specifying them as arguments: sed 's/old/new/' file1 file2.

17. Remove Leading Whitespace:

    • To remove leading whitespace, use sed 's/^[ \t]*//' file.

18. Remove Trailing Whitespace:

    • To remove trailing whitespace, use sed 's/[ \t]*$//' file.

sed is a versatile tool for text processing and manipulation, and it's commonly used in scripts and command pipelines for various tasks. As you gain experience, you'll discover its many powerful features for text editing and transformation.

 

 

1. Basic Structure:

   • grep pattern file searches for lines in file that match the specified pattern.

2. Case-Insensitive Search:

   • Use the -i option to perform a case-insensitive search. For example, grep -i pattern file matches "Pattern," "PATTERN," and "pattern."

3. Whole Word Search:

   • Use the -w option to search for whole words only. For example, grep -w word file matches "word" but not "words" or "sword."

4. Invert Match:

   • Use the -v option to invert the match, showing lines that don't contain the pattern. For example, grep -v pattern file shows lines without "pattern."

5. Count Matches:

   • Use the -c option to count the number of matches. For example, grep -c pattern file counts how many times "pattern" appears.

6. Regular Expressions:

   • grep supports regular expressions for complex pattern matching. For example, grep '^a.*z$' file matches lines that start with "a" and end with "z."

7. Print Line Numbers:

   • Use the -n option to print line numbers along with matching lines. For example, grep -n pattern file shows line numbers.

8. Recursive Search:

   • Search for a pattern in all files under a directory and its subdirectories using the -r option. For example, grep -r pattern directory.

9. Show Context:

   • Use the -A, -B, or -C options to display context around the match. For example, grep -C 2 pattern file shows two lines of context around each match.

10. Filter Files by Type:

    • Use the --include option to specify file types to search. For example, grep pattern --include='*.txt' directory searches for "pattern" only in text files.

11. Multiple Patterns:

    • Search for lines that match multiple patterns using the | (pipe) character. For example, grep 'pattern1\|pattern2' file matches lines with either "pattern1" or "pattern2."

12. Quiet Mode:

    • Use the -q option for quiet mode, which is useful for scripting. It doesn't display matching lines but returns an exit code indicating success or failure.

13. Fixed Strings:

    • Use the -F option to treat the pattern as a fixed string (no regular expressions). This is useful when searching for special characters.

14. Print Matching Patterns Only:

    • Use the -o option to print only the matching part of the line, rather than the whole line.

"Grep" is a versatile tool for searching and pattern matching in text files. It's widely used in Unix-like systems and is a valuable tool for text processing and data extraction.

 

find 

  The find command is a powerful tool for searching and locating files and directories within a Unix-like file system. It offers a wide range of options for fine-tuned searches.

1. Basic Syntax:

   • find <path> <options> <pattern>: Searches for files and directories in the specified <path>.

2. Search by Name:

   • To find files with a specific name, use find <path> -name "<filename>". For example, find /home/user -name "file.txt".

3. Search by Type:

   • To find files or directories of a specific type, use -type. For example, find /path -type f finds files, and find /path -type d finds directories.

4. Search by Size:

   • To find files by size, use -size. For example, find /path -size +1M finds files larger than 1MB.

5. Search by Modification Time:

   • Use -mtime to find files modified in a specific number of days. For example, find /path -mtime -7 finds files modified within the last 7 days.

6. Search by Access Time:

   • Use -atime to find files accessed in a specific number of days. For example, find /path -atime +30 finds files accessed more than 30 days ago.

7. Search by Owner:

   • To find files owned by a specific user, use -user. For example, find /path -user username.

8. Search by Group:

   • To find files in a specific group, use -group. For example, find /path -group groupname.

9. Search by Permissions:

   • To find files with specific permissions, use -perm. For example, find /path -perm 644 finds files with read and write permissions for the owner.

10. Search for Empty Files and Directories:

    • To find empty files or directories, use -empty. For example, find /path -type d -empty finds empty directories.

11. Search with Logical Operators:

    • Combine search criteria with logical operators like -and, -or, and -not. For example, find /path -name "*.txt" -or -name "*.log" finds files with either .txt or .log extensions.

12. Execute Commands on Results:

    • Use -exec to perform actions on the search results. For example, find /path -name "*.txt" -exec rm {} \; deletes all .txt files.

13. Limit the Depth of Search:

    • Use -maxdepth to limit the depth of the search. For example, find /path -maxdepth 2 searches up to a depth of 2 directories.

14. Print Results:

    • By default, find prints the results to the standard output. You can redirect the output to a file for further analysis.

15. Search Multiple Directories:

    • You can specify multiple paths to search in different directories.

16. Case-Insensitive Search:

    • Use the -iname option for a case-insensitive search.

17. Save Results to a File:

    • Use the > or >> operators to save the results to a file. For example, find /path -name "*.txt" > results.txt.

"find" is a versatile and powerful tool for locating files and directories on a Unix-like system. By mastering its various options and combining them, you can perform advanced and precise file searches.

 

 

xargs is a command that allows you to process input and execute commands with that input, which can be particularly useful when working with lists of items or data from standard input.

1. Basic Syntax:

   • xargs [options] [command]: xargs reads items from standard input (usually separated by spaces or newlines) and passes them as arguments to the specified command.

2. Reading from Standard Input:

   • By default, xargs reads data from standard input (e.g., via a pipe). For example, echo "file1 file2 file3" | xargs command passes "file1," "file2," and "file3" as arguments to command.

3. Passing Arguments:

   • The command is the action to perform on each item received. For example, xargs -I {} mv {} /destination moves each item to the specified destination.

4. Specifying Delimiters:

   • Use the -d or --delimiter option to specify a custom delimiter for input items. For example, echo "item1:item2:item3" | xargs -d ':' command splits the input on colons.

5. Maximum Arguments:

   • Use the -n or --max-args option to limit the number of arguments passed to each command. For example, echo "file1 file2 file3" | xargs -n 2 command processes two items at a time.

6. Interactive Mode:

   • The -p option prompts the user for confirmation before executing each command. For example, find /path -type f | xargs -p rm asks for confirmation before removing each file.

7. Verbose Mode:

   • Use the -t or --verbose option to print the command to be executed before running it. For example, echo "file1 file2" | xargs -t -I {} mv {} /destination shows the mv command before execution.

8. Substituting with Placeholder:

   • The -I option allows you to specify a placeholder (e.g., {}) to represent the input item in the command. For example, echo "file1 file2" | xargs -I {} mv {} /destination moves files to the destination.

9. Parallel Execution:

   • xargs can execute commands in parallel using the -P option, followed by the maximum number of parallel processes. For example, echo "file1 file2" | xargs -P 2 -I {} command runs two command processes concurrently.

10. Using a File List:

    • You can read input from a file by using the -a or --arg-file option. For example, xargs -a input.txt command processes items listed in input.txt.

11. Ignoring Empty Lines:

    • Use the -r or --no-run-if-empty option to prevent command execution if no input items are provided.

12. Combining with Find:

    • find can be used in combination with xargs to execute commands on files found by find. For example, find /path -type f | xargs command executes command on each file found.

xargs is a versatile tool for processing lists of items or data from standard input and applying commands to them efficiently. It's commonly used in conjunction with other commands to perform batch operations and streamline text processing tasks.

 

credits

chatgpt